Define semantic convention group and mixed stability rules (#1472)

.chloggen/1472.yaml

@@ -0,0 +1,4 @@
+change_type: enhancement
+component: http
+note: Define how to handle experimental attributes in stable groups, add policies and move experimental HTTP attributes into stable HTTP groups (as opt_in).
+issues: [906, 1472]

docs/general/group-stability.md

@@ -0,0 +1,90 @@
+<!--- Hugo front matter used to generate the website version of this page:
+linkTitle: Semantic Convention Groups
+aliases: [semantic-convention-groups]
+--->
+
+# Semantic Convention Groups
+
+**Status**: [Experimental][DocumentStatus]
+
+Spans, metrics, events, and resources are defined in semantic convention groups in YAML schema.
+Each group has a `type` property that could be one of the following:
+
+- `span` - defines semantic convention for a specific type of span, such as HTTP `CLIENT`
+- `metric` - defines semantic convention for a specific metric, such as HTTP client request duration
+- `event` - defines semantic conventions for a specific event, such as exception.
+- `resource` - defines semantic conventions for a specific entity the telemetry is collected within,
+  such as `service`.
+
+Groups that have `attribute_group` type do not describe semantic convention and
+are used for auxiliary purposes.
+
+All semantic convention groups have the following common properties:
+
+- `id` - identifies specific span type, metric instrument, or event
+  among other spans, instruments, or events.
+- `brief` and `note` provide human-readable description of the convention
+- `stability` describes the maturity level of the convention
+- `deprecated` property marks convention as deprecated and provides information about
+  replacement or other details.
+- `attributes` lists references to applicable attributes in the [registry](../attributes-registry/README.md)
+
+In addition to common properties, semantic convention groups have type-specific properties, see
+[Schema documentation](https://github.com/open-telemetry/weaver/blob/main/schemas/semconv-syntax.md)
+for the details.
+
+## Group Stability
+
+<!-- TODO: this section will need to change when https://github.com/open-telemetry/semantic-conventions/issues/1096 is implemented -->
+
+Semantic Convention groups can be `stable` (corresponds to
+[Stable maturity level][MaturityLevel]) or `experimental` (corresponds to [Development maturity level][MaturityLevel])
+if stability level is not specified, it's assumed to be `experimental`.
+
+Group stability MUST NOT change from `stable` to `experimental`.
+
+Semantic convention group of any stability level MUST NOT be removed
+to preserve code generation and documentation for legacy instrumentations.
+
+When group is renamed or no longer recommended, it SHOULD be deprecated.
+
+See [Versioning and Stability][Stability] for the details on stability guarantees
+provided for semantic convention groups of different types.
+
+Stability guarantees **do not** apply to groups with `attribute_group` type as they
+don't describe telemetry items.
+
+### Groups with mixed stability
+
+Stability guarantees on a group apply to the group properties (such as type, id and
+signal-specific properties) as well as overridden properties of stable attributes
+referenced by this group.
+
+Stability guarantees on a group level **do not** apply to experimental attribute references.
+
+**Experimental groups:**
+
+- MAY add or remove references to stable or experimental attributes
+- MAY change requirement level and other properties of attribute references
+
+**Stable groups:**
+
+- MAY add or remove references to experimental attributes with `opt_in`
+  requirement level.
+- SHOULD NOT have references to experimental attributes with requirement level
+  other than `opt_in`.
+  The requirement level of an experimental attribute reference
+  MAY be changed when this attribute becomes stable in cases allowed by the
+  [Versioning and Stability][Stability].
+- MUST NOT remove references to stable attributes.
+
+Stable instrumentations MUST NOT report telemetry following the experimental part
+of semantic conventions by default. They MAY support experimental part and allow
+users to opt into it.
+
+<!-- TODO: SchemaURL needs to contain some indication of stability level, e.g. as a suffix -->
+<!-- https://github.com/open-telemetry/semantic-conventions/issues/1511 -->
+
+[Stability]: https://opentelemetry.io/docs/specs/otel/versioning-and-stability/#semantic-conventions-stability
+[MaturityLevel]: https://github.com/open-telemetry/oteps/blob/main/text/0232-maturity-of-otel.md
+[DocumentStatus]: https://opentelemetry.io/docs/specs/otel/document-status

docs/http/http-metrics.md

@@ -21,7 +21,6 @@ operations. By adding HTTP attributes to metric events it allows for finely tune
   - [Metric: `http.server.response.body.size`](#metric-httpserverresponsebodysize)
 - [HTTP Client](#http-client)
   - [Metric: `http.client.request.duration`](#metric-httpclientrequestduration)
-    - [Experimental attributes](#experimental-attributes)
   - [Metric: `http.client.request.body.size`](#metric-httpclientrequestbodysize)
   - [Metric: `http.client.response.body.size`](#metric-httpclientresponsebodysize)
   - [Metric: `http.client.open_connections`](#metric-httpclientopen_connections)
@@ -486,6 +485,7 @@ of `[ 0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10
 | [`network.protocol.name`](/docs/attributes-registry/network.md) | string | [OSI application layer](https://osi-model.com/application-layer/) or non-OSI equivalent. [5] | `http`; `spdy` | `Conditionally Required` [6] | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
 | [`network.protocol.version`](/docs/attributes-registry/network.md) | string | The actual version of the protocol used for network communication. [7] | `1.0`; `1.1`; `2`; `3` | `Recommended` | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
 | [`url.scheme`](/docs/attributes-registry/url.md) | string | The [URI scheme](https://www.rfc-editor.org/rfc/rfc3986#section-3.1) component identifying the used protocol. | `http`; `https` | `Opt-In` | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
+| [`url.template`](/docs/attributes-registry/url.md) | string | The low-cardinality template of an [absolute path reference](https://www.rfc-editor.org/rfc/rfc3986#section-4.2). [8] | `/users/{id}`; `/users/:id`; `/users?id={id}` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
 
 **[1]:** HTTP request method value SHOULD be "known" to the instrumentation.
 By default, this convention defines "known" methods as the ones listed in [RFC9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-methods)
@@ -529,6 +529,8 @@ If the request has completed successfully, instrumentations SHOULD NOT set `erro
 
 **[7]:** If protocol version is subject to negotiation (for example using [ALPN](https://www.rfc-editor.org/rfc/rfc7301.html)), this attribute SHOULD be set to the negotiated version. If the actual protocol version is not known, this attribute SHOULD NOT be set.
 
+**[8]:** The `url.template` MUST have low cardinality. It is not usually available on HTTP clients, but may be known by the application or specialized HTTP instrumentation.
+
 `error.type` has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.
 
 | Value  | Description | Stability |
@@ -555,30 +557,6 @@ If the request has completed successfully, instrumentations SHOULD NOT set `erro
 <!-- END AUTOGENERATED TEXT -->
 <!-- endsemconv -->
 
-#### Experimental attributes
-
-**Status**: [Experimental][DocumentStatus]
-
-Instrumentations MAY allow users to enable additional experimental attributes.
-
-<!-- semconv attributes.http.client.experimental -->
-<!-- NOTE: THIS TEXT IS AUTOGENERATED. DO NOT EDIT BY HAND. -->
-<!-- see templates/registry/markdown/snippet.md.j2 -->
-<!-- prettier-ignore-start -->
-<!-- markdownlint-capture -->
-<!-- markdownlint-disable -->
-
-| Attribute  | Type | Description  | Examples  | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) | Stability |
-|---|---|---|---|---|---|
-| [`url.template`](/docs/attributes-registry/url.md) | string | The low-cardinality template of an [absolute path reference](https://www.rfc-editor.org/rfc/rfc3986#section-4.2). [1] | `/users/{id}`; `/users/:id`; `/users?id={id}` | `Opt-In` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
-
-**[1]:** The `url.template` MUST have low cardinality. It is not usually available on HTTP clients, but may be known by the application or specialized HTTP instrumentation.
-
-<!-- markdownlint-restore -->
-<!-- prettier-ignore-end -->
-<!-- END AUTOGENERATED TEXT -->
-<!-- endsemconv -->
-
 ### Metric: `http.client.request.body.size`
 
 This metric is optional.