Skip to content

Commit

Permalink
Introduce gen_ai.request.seed as needed by cohere
Browse files Browse the repository at this point in the history
And deprecate gen_ai.openai.request_seed.
  • Loading branch information
xrmx committed Dec 23, 2024
1 parent 07e99eb commit 2f4fdfb
Show file tree
Hide file tree
Showing 4 changed files with 26 additions and 7 deletions.
11 changes: 10 additions & 1 deletion docs/attributes-registry/gen-ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- [GenAI Attributes](#genai-attributes)
- [OpenAI Attributes](#openai-attributes)
- [Deprecated GenAI Attributes](#deprecated-genai-attributes)
- [Deprecated OpenAI GenAI Attributes](#deprecated-openai-genai-attributes)

## GenAI Attributes

Expand All @@ -22,6 +23,7 @@ This document defines the attributes used to describe telemetry in the context o
| <a id="gen-ai-request-max-tokens" href="#gen-ai-request-max-tokens">`gen_ai.request.max_tokens`</a> | int | The maximum number of tokens the model generates for a request. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-model" href="#gen-ai-request-model">`gen_ai.request.model`</a> | string | The name of the GenAI model a request is being made to. | `gpt-4` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-presence-penalty" href="#gen-ai-request-presence-penalty">`gen_ai.request.presence_penalty`</a> | double | The presence penalty setting for the GenAI request. | `0.1` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-seed" href="#gen-ai-request-seed">`gen_ai.request.seed`</a> | int | Requests with same seed value more likely to return same result. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-stop-sequences" href="#gen-ai-request-stop-sequences">`gen_ai.request.stop_sequences`</a> | string[] | List of sequences that the model will use to stop generating further tokens. | `["forest", "lived"]` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-temperature" href="#gen-ai-request-temperature">`gen_ai.request.temperature`</a> | double | The temperature setting for the GenAI request. | `0.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-request-top-k" href="#gen-ai-request-top-k">`gen_ai.request.top_k`</a> | double | The top_k sampling setting for the GenAI request. | `1.0` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand Down Expand Up @@ -88,7 +90,6 @@ Thie group defines attributes for OpenAI.
| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="gen-ai-openai-request-response-format" href="#gen-ai-openai-request-response-format">`gen_ai.openai.request.response_format`</a> | string | The response format that is requested. | `json` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-request-seed" href="#gen-ai-openai-request-seed">`gen_ai.openai.request.seed`</a> | int | Requests with same seed value more likely to return same result. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-request-service-tier" href="#gen-ai-openai-request-service-tier">`gen_ai.openai.request.service_tier`</a> | string | The service tier requested. May be a specific tier, default, or auto. | `auto`; `default` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-response-service-tier" href="#gen-ai-openai-response-service-tier">`gen_ai.openai.response.service_tier`</a> | string | The service tier used for the response. | `scale`; `default` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| <a id="gen-ai-openai-response-system-fingerprint" href="#gen-ai-openai-response-system-fingerprint">`gen_ai.openai.response.system_fingerprint`</a> | string | A fingerprint to track any eventual change in the Generative AI environment. | `fp_44709d6fcb` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
Expand Down Expand Up @@ -122,3 +123,11 @@ Describes deprecated `gen_ai` attributes.
| <a id="gen-ai-prompt" href="#gen-ai-prompt">`gen_ai.prompt`</a> | string | Deprecated, use Event API to report prompt contents. | `[{'role': 'user', 'content': 'What is the capital of France?'}]` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Removed, no replacement at this time. |
| <a id="gen-ai-usage-completion-tokens" href="#gen-ai-usage-completion-tokens">`gen_ai.usage.completion_tokens`</a> | int | Deprecated, use `gen_ai.usage.output_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.usage.output_tokens` attribute. |
| <a id="gen-ai-usage-prompt-tokens" href="#gen-ai-usage-prompt-tokens">`gen_ai.usage.prompt_tokens`</a> | int | Deprecated, use `gen_ai.usage.input_tokens` instead. | `42` | ![Deprecated](https://img.shields.io/badge/-deprecated-red)<br>Replaced by `gen_ai.usage.input_tokens` attribute. |

## Deprecated OpenAI GenAI Attributes

Describes deprecated `gen_ai.openai` attributes.

| Attribute | Type | Description | Examples | Stability |
|---|---|---|---|---|
| <a id="gen-ai-openai-request-seed" href="#gen-ai-openai-request-seed">`gen_ai.openai.request.seed`</a> | int | Deprecated, use `gen_ai.request.seed`. | `100` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
2 changes: 1 addition & 1 deletion docs/gen-ai/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ attributes and ones specific the OpenAI.
| [`gen_ai.request.model`](/docs/attributes-registry/gen-ai.md) | string | The name of the GenAI model a request is being made to. [2] | `gpt-4` | `Required` | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`error.type`](/docs/attributes-registry/error.md) | string | Describes a class of error the operation ended with. [3] | `timeout`; `java.net.UnknownHostException`; `server_certificate_invalid`; `500` | `Conditionally Required` if the operation ended in an error | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
| [`gen_ai.openai.request.response_format`](/docs/attributes-registry/gen-ai.md) | string | The response format that is requested. | `json` | `Conditionally Required` if the request includes a response_format | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Requests with same seed value more likely to return same result. | `100` | `Conditionally Required` if the request includes a seed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.request.seed`](/docs/attributes-registry/gen-ai.md) | int | Deprecated, use `gen_ai.request.seed`. | `100` | `Conditionally Required` if the request includes a seed | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.request.service_tier`](/docs/attributes-registry/gen-ai.md) | string | The service tier requested. May be a specific tier, default, or auto. | `auto`; `default` | `Conditionally Required` [4] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`gen_ai.openai.response.service_tier`](/docs/attributes-registry/gen-ai.md) | string | The service tier used for the response. | `scale`; `default` | `Conditionally Required` [5] | ![Experimental](https://img.shields.io/badge/-experimental-blue) |
| [`server.port`](/docs/attributes-registry/server.md) | int | GenAI server port. [6] | `80`; `8080`; `443` | `Conditionally Required` If `server.address` is set. | ![Stable](https://img.shields.io/badge/-stable-lightgreen) |
Expand Down
10 changes: 10 additions & 0 deletions model/gen-ai/deprecated/registry-deprecated.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,13 @@ groups:
deprecated: "Removed, no replacement at this time."
brief: "Deprecated, use Event API to report completions contents."
examples: ["[{'role': 'assistant', 'content': 'The capital of France is Paris.'}]"]
- id: registry.gen_ai.openai.deprecated
type: attribute_group
brief: Describes deprecated `gen_ai.openai` attributes.
display_name: Deprecated OpenAI GenAI Attributes
attributes:
- id: gen_ai.openai.request.seed
stability: experimental
type: int
brief: "Deprecated, use `gen_ai.request.seed`."
examples: [100]
10 changes: 5 additions & 5 deletions model/gen-ai/registry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ groups:
note: >
In some GenAI systems the encoding formats are called embedding types.
Also, some GenAI systems only accept a single format per request.
- id: gen_ai.request.seed
stability: experimental
type: int
brief: Requests with same seed value more likely to return same result.
examples: [100]
- id: gen_ai.response.id
stability: experimental
type: string
Expand Down Expand Up @@ -168,11 +173,6 @@ groups:
brief: >
Thie group defines attributes for OpenAI.
attributes:
- id: gen_ai.openai.request.seed
stability: experimental
type: int
brief: Requests with same seed value more likely to return same result.
examples: [100]
- id: gen_ai.openai.request.response_format
stability: experimental
type:
Expand Down

0 comments on commit 2f4fdfb

Please sign in to comment.