Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CICD metrics #1681

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
72b225b
[cicd] add pipeline run duration metric
christophe-kamphaus-jemmic Dec 13, 2024
1b10459
[cicd] add cicd queue metrics
christophe-kamphaus-jemmic Dec 13, 2024
d820586
[cicd] add cicd worker count metric
christophe-kamphaus-jemmic Dec 13, 2024
a2edc16
[cicd] add cicd error count
christophe-kamphaus-jemmic Dec 13, 2024
3e1537b
Update vscode settings to align with markdown-toc --no-first-h1
christophe-kamphaus-jemmic Dec 13, 2024
7d4affa
[cicd] update examples of cicd.pipeline.result
christophe-kamphaus-jemmic Dec 13, 2024
70f2bd6
[cicd] add changelog entry
christophe-kamphaus-jemmic Dec 13, 2024
1c4e584
[cicd] update brief to add missing article
christophe-kamphaus-jemmic Dec 13, 2024
2f9b857
[cicd] improve metric brief
christophe-kamphaus-jemmic Dec 13, 2024
65b854f
Merge branch 'main' into 1600-cicd-metrics
christophe-kamphaus-jemmic Dec 14, 2024
a044341
[cicd] add skipped as possible cicd.pipeline.result value
christophe-kamphaus-jemmic Dec 14, 2024
7330043
[cicd] add down as possible cicd.worker.state value
christophe-kamphaus-jemmic Dec 14, 2024
a243084
[cicd] improve brief and add note for idle cicd.worker.state
christophe-kamphaus-jemmic Dec 14, 2024
3a31dd9
[cicd] add metric cicd.pipeline.run.executing
christophe-kamphaus-jemmic Dec 14, 2024
66732c5
[cicd] add cicd.worker.type container
christophe-kamphaus-jemmic Dec 16, 2024
2e8ab02
[cicd] mark all cicd metrics as recommended
christophe-kamphaus-jemmic Dec 16, 2024
1562dc6
Merge branch 'main' into 1600-cicd-metrics
christophe-kamphaus-jemmic Dec 25, 2024
dc09536
[cicd] improve brief of `cicd.worker.count` metric
christophe-kamphaus-jemmic Dec 25, 2024
26d3064
[cicd] renamed attribute `cicd.worker.type` to `cicd.worker.class`
christophe-kamphaus-jemmic Dec 25, 2024
a2202cd
[cicd] adapt metric brief following attribute rename
christophe-kamphaus-jemmic Dec 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .chloggen/1600-cicd-metrics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: cicd

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add CICD metrics

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [1600]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
Makes the following changes:

- Add metrics `cicd.pipeline.run.duration`, `cicd.pipeline.run.executing`, `cicd.queue.latency`, `cicd.queue.length`, `cicd.worker.count`, `cicd.errors`.
- The CICD attributes `cicd.pipeline.result`, `cicd.worker.state` and `cicd.worker.type` have been added to the registry.
3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@
"model/**/*.yaml"
]
},
"json.schemaDownload.enable": true
"json.schemaDownload.enable": true,
"markdown.extension.toc.levels": "2..6"
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
}
37 changes: 37 additions & 0 deletions docs/attributes-registry/cicd.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,27 @@ This group describes attributes specific to pipelines within a Continuous Integr
| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="cicd-pipeline-name" href="#cicd-pipeline-name">`cicd.pipeline.name`</a> | string | The human readable name of the pipeline within a CI/CD system. | `Build and Test`; `Lint`; `Deploy Go Project`; `deploy_to_environment` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-result" href="#cicd-pipeline-result">`cicd.pipeline.result`</a> | string | The result of a pipeline run. | `success`; `failure`; `timeout`; `skipped` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-run-id" href="#cicd-pipeline-run-id">`cicd.pipeline.run.id`</a> | string | The unique identifier of a pipeline run within a CI/CD system. | `120912` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-task-name" href="#cicd-pipeline-task-name">`cicd.pipeline.task.name`</a> | string | The human readable name of a task within a pipeline. Task here most closely aligns with a [computing process](https://wikipedia.org/wiki/Pipeline_(computing)) in a pipeline. Other terms for tasks include commands, steps, and procedures. | `Run GoLang Linter`; `Go Build`; `go-test`; `deploy_binary` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-task-run-id" href="#cicd-pipeline-task-run-id">`cicd.pipeline.task.run.id`</a> | string | The unique identifier of a task run within a pipeline. | `12097` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-task-run-url-full" href="#cicd-pipeline-task-run-url-full">`cicd.pipeline.task.run.url.full`</a> | string | The [URL](https://wikipedia.org/wiki/URL) of the pipeline run providing the complete address in order to locate and identify the pipeline run. | `https://github.com/open-telemetry/semantic-conventions/actions/runs/9753949763/job/26920038674?pr=1075` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-pipeline-task-type" href="#cicd-pipeline-task-type">`cicd.pipeline.task.type`</a> | string | The type of the task within a pipeline. | `build`; `test`; `deploy` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-worker-state" href="#cicd-worker-state">`cicd.worker.state`</a> | string | The state of a CICD worker / agent. | `idle`; `busy`; `down` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="cicd-worker-type" href="#cicd-worker-type">`cicd.worker.type`</a> | string | The type of worker / agent used by the CICD system. | `vm`; `pod` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved

---

`cicd.pipeline.result` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

| Value | Description | Stability |
|---|---|---|
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved
| `cancelled` | The pipeline run was cancelled, eg. by a user manually cancelling the pipeline run. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `error` | The pipeline run failed due to an error in the CICD system, eg. due to the worker being killed. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `failure` | The pipeline run did not finish successfully, eg. due to a compile error or a failing test. Such failures are usually detected by non-zero exit codes of the tools executed in the pipeline run. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `skipped` | The pipeline run was skipped, eg. due to a precondition not being met. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `success` | The pipeline run finished successfully. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `timeout` | A timeout caused the pipeline run to be interrupted. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

---

Expand All @@ -28,3 +44,24 @@ This group describes attributes specific to pipelines within a Continuous Integr
| `build` | build | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `deploy` | deploy | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `test` | test | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

---

`cicd.worker.state` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved

| Value | Description | Stability |
|---|---|---|
| `busy` | The worker is performing work for the CICD system. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `down` | The worker is not available to the CICD system (disconnected / down). | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `idle` | The worker is not performing work for the CICD system. It is available to the CICD system to perform work on. [1] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |

**[1]:** Pipelines might have conditions on which workers they are able to run so not every worker might be available to every pipeline.

---

`cicd.worker.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
christophe-kamphaus-jemmic marked this conversation as resolved.
Show resolved Hide resolved

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In many complex CI/CDs the workers are categorized by other attributes, such as the platform they run (ubuntu, windows) the tooling they have installed (java, go) or other attributes.

I lean towards leaving this attribute open (without pre-defined values). Also consider class as an alternative name to reflect its intention to group the workers into categories.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have rename this attribute to cicd.worker.class in 26d3064.


| Value | Description | Stability |
|---|---|---|
| `pod` | One or more containers deployed together. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| `vm` | A virtual machine or baremetal host. | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Loading
Loading