Feat: OTEL with Dynatrace + plugin doc refactor (#7881)

* dynatrace with otel doc * refactor otel docs to fit structure * add more search terms * make vale happy * vale and cleanup * add dynatrace to dictionary * Apply suggestions from code review Co-authored-by: Angel <[email protected]> --------- Co-authored-by: Angel <[email protected]>
Kong · Sep 11, 2024 · ab90839 · ab90839
1 parent d12ec2f
commit ab90839
Show file tree

Hide file tree

Showing 6 changed files with 380 additions and 198 deletions.
diff --git a/.github/styles/kong/dictionary.txt b/.github/styles/kong/dictionary.txt
@@ -34,6 +34,7 @@ DNs
 DPP
 DPPs
 Datadog
+Dynatrace
 Dex
 ElastiCache
 Entra

diff --git a/app/_hub/kong-inc/opentelemetry/_metadata/_index.yml b/app/_hub/kong-inc/opentelemetry/_metadata/_index.yml
@@ -3,6 +3,11 @@ search_aliases:
   - otlp
   - otel
   - open telemetry
+  - dynatrace
+  - tracing
+  - logging
+  - analytics
+  - monitoring
 dbless_compatible: 'yes'
 free: true
 enterprise: true
@@ -12,5 +17,4 @@ notes: --
 categories:
   - analytics-monitoring
 publisher: Kong Inc.
-version: 0.1.0
 desc: Propagate spans and report space to a backend server through OTLP protocol.
diff --git a/app/_hub/kong-inc/opentelemetry/how-to/_dynatrace.md b/app/_hub/kong-inc/opentelemetry/how-to/_dynatrace.md
@@ -0,0 +1,115 @@
+---
+title: Set up Dynatrace with OpenTelemetry
+nav_title: Using Dynatrace
+minimum_version: 3.8.x
+---
+
+Set up the OpenTelemetry plugin to send logs and metrics to Dynatrace.
+
+## Prerequisites
+* Install the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/installation/)
+* {{site.base_gateway}} 3.8+
+
+## Configure {{site.base_gateway}}
+
+Set the following parameters in your [`kong.conf`](/gateway/latest/production/kong-conf/) file:
+
+```
+tracing_instrumentations = all
+tracing_sampling_rate = 1.0
+```
+
+## Configure the OpenTelemetry plugin
+
+Adjust the `{OPENTELEMETRY_COLLECTOR}` variable with your own collector endpoint:
+
+<!--vale off-->
+{% plugin_example %}
+plugin: kong-inc/opentelemetry
+name: opentelemetry
+config:
+  traces_endpoint: "http://{OPENTELEMETRY_COLLECTOR}:4318/v1/traces"
+  logs_endpoint: "http://{OPENTELEMETRY_COLLECTOR}:4318/v1/logs"
+  resource_attributes:
+    service.name: kong-dev
+targets:
+  - service
+  - route
+  - consumer
+  - global
+formats:
+  - curl
+  - konnect
+  - yaml
+  - kubernetes
+{% endplugin_example %}
+<!--vale on-->
+
+## Configure the OpenTelemetry Collector
+
+Configure your OpenTelemetry Collector to send data to the Dynatrace environment. 
+
+The following example OpenTelemetry configuration shows how to export traces and logs:
+
+```yaml
+receivers:
+  otlp:
+    protocols:
+      http:
+        endpoint: 0.0.0.0:4318
+
+exporters:
+  otlphttp:
+    endpoint: "${env:DT_BASEURL}/api/v2/otlp"
+    headers:
+      "Authorization": "Api-Token ${env:DT_API_TOKEN}"
+
+service:
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: []
+      exporters: [otlphttp]
+    logs:
+      receivers: [otlp]
+      processors: []
+      exporters: [otlphttp]
+```
+
+## Export application span metrics
+
+To include span metrics for application traces, configure the collector exporters section of 
+the OpenTelemetry Collector configuration file: 
+
+```yaml
+connectors:
+  spanmetrics:
+    dimensions:
+      - name: http.method
+        default: GET
+      - name: http.status_code
+      - name: http.route
+    exclude_dimensions:
+      - status.code
+    metrics_flush_interval: 15s
+    histogram:
+      disable: false
+
+service:
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: []
+      exporters: [spanmetrics]
+    metrics:
+      receivers: [spanmetrics]
+      processors: []
+      exporters: [otlphttp]
+```
+
+## More information
+
+* [Troubleshooting](/hub/kong-inc/opentelemetry/#troubleshooting)
+* [How logging works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#logging)
+* [How tracing works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#tracing)
+* [Dynatrace documentation](https://docs.dynatrace.com/docs/setup-and-configuration/technology-support/application-software/nginx/kong-gateway#kong-observability-opentelemetry)
diff --git a/app/_hub/kong-inc/opentelemetry/how-to/_getting-started.md b/app/_hub/kong-inc/opentelemetry/how-to/_getting-started.md
@@ -0,0 +1,167 @@
+---
+title: Get Started with the OpenTelemetry plugin
+nav_title: Get Started with the OpenTelemetry plugin
+---
+
+{% if_version gte:3.2.x %}
+{:.note}
+> **Note**: The OpenTelemetry plugin's tracing capabilities only work when {{site.base_gateway}}'s `tracing_instrumentations` configuration is enabled.
+{% endif_version %}
+
+{% if_version lte:3.1.x %}
+{:.note}
+> **Note**: The OpenTelemetry plugin only works when {{site.base_gateway}}'s `opentelemetry_tracing` configuration is enabled.
+{% endif_version %}
+
+The OpenTelemetry plugin is fully compatible with the OpenTelemetry specification and can be used with any OpenTelemetry compatible backend.
+
+There are two ways to set up an OpenTelemetry backend:
+* Using a [OpenTelemetry compatible backend](#set-up-an-opentelemetry-compatible-backend) directly, like Jaeger (v1.35.0+)
+   All the vendors supported by OpenTelemetry are listed in the [OpenTelemetry's Vendor support](https://opentelemetry.io/vendors/).
+* Using the [OpenTelemetry Collector](#set-up-an-opentelemetry-collector), which is middleware that can be used to proxy OpenTelemetry spans to a compatible backend.
+   You can view all the available OpenTelemetry Collector exporters at [open-telemetry/opentelemetry-collector-contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter).
+
+## Set up {{site.base_gateway}}
+
+The OpenTelemetry tracing capability supported by this plugin requires the following {{site.base_gateway}} configuration:
+
+{% if_version lte:3.1.x %}
+- `opentelemetry_tracing = all`: Enables all possible tracing instrumentations. 
+Valid values can be found in [Kong's configuration reference](/gateway/latest/reference/configuration/#tracing_instrumentations).
+- `opentelemetry_tracing_sampling_rate = 1.0`: Tracing instrumentation sampling rate.
+  Tracer samples a fixed percentage of all spans following the sampling rate.
+  Set the sampling rate to a lower value to reduce the impact of the instrumentation on {{site.base_gateway}}'s proxy performance in production.
+{% endif_version %}
+{% if_version gte:3.2.x %}
+- `tracing_instrumentations = all`: Enables all possible tracing instrumentations. 
+Valid values can be found in [Kong's configuration reference](/gateway/latest/reference/configuration/#tracing_instrumentations).
+- `tracing_sampling_rate = 1.0`: Tracing instrumentation sampling rate.
+  Tracer samples a fixed percentage of all spans following the sampling rate.
+  Set the sampling rate to a lower value to reduce the impact of the instrumentation on {{site.base_gateway}}'s proxy performance in production.
+{% endif_version %}
+
+## Set up an OpenTelemetry compatible backend
+
+This section is optional if you are using a OpenTelemetry compatible APM vendor.
+All the supported vendors are listed in the [OpenTelemetry's Vendor support](https://opentelemetry.io/vendors/).
+
+Jaeger [natively supports OpenTelemetry](https://www.jaegertracing.io/docs/features/#native-support-for-opentracing-and-opentelemetry) starting with v1.35 and can be used with the OpenTelemetry plugin.
+
+Deploy a Jaeger instance with Docker:
+
+```bash
+docker run --name jaeger \
+  -e COLLECTOR_OTLP_ENABLED=true \
+  -p 16686:16686 \
+  -p 4317:4317 \
+  -p 4318:4318 \
+  jaegertracing/all-in-one:1.36
+```
+
+* The `COLLECTOR_OTLP_ENABLED` environment variable must be set to `true` to enable the OpenTelemetry Collector.
+
+* The `4318` port is the OTLP/HTTP port and the `4317` port is the OTLP/GRPC port that isn't supported by the OpenTelemetry plugin yet.
+
+## Set up an OpenTelemetry Collector
+
+This section is required if you are using an incompatible OpenTelemetry APM vendor.
+
+Create a config file (`otelcol.yaml`) for the OpenTelemetry Collector:
+
+```yaml
+receivers:
+  otlp:
+    protocols:
+      grpc:
+      http:
+
+processors:
+  batch:
+
+exporters:
+  logging:
+    loglevel: debug
+  zipkin:
+    endpoint: "http://some.url:9411/api/v2/spans"
+    tls:
+      insecure: true
+
+service:
+  pipelines:
+    traces:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [logging, zipkin]
+    {% if_version gte:3.8.x %}logs:
+      receivers: [otlp]
+      processors: [batch]
+      exporters: [logging]{% endif_version %}
+```
+
+Run the OpenTelemetry Collector with Docker:
+
+```bash
+docker run --name opentelemetry-collector \
+  -p 4317:4317 \
+  -p 4318:4318 \
+  -p 55679:55679 \
+  -v $(pwd)/otelcol.yaml:/etc/otel-collector-config.yaml \
+  otel/opentelemetry-collector-contrib:0.52.0 \
+  --config=/etc/otel-collector-config.yaml
+```
+
+See the [OpenTelemetry Collector documentation](https://opentelemetry.io/docs/collector/configuration/) for more information.
+
+## Configure the OpenTelemetry plugin
+
+<!--vale off-->
+{% if_version lte:3.7.x %}
+{% plugin_example %}
+plugin: kong-inc/opentelemetry
+name: opentelemetry
+config:
+  endpoint: "http://<opentelemetry-backend>:4318/v1/traces"
+  resource_attributes:
+    service.name: kong-dev
+targets:
+  - service
+  - route
+  - consumer
+  - global
+formats:
+  - curl
+  - konnect
+  - yaml
+  - kubernetes
+{% endplugin_example %}
+{% endif_version %}
+
+{% if_version gte:3.8.x %}
+{% plugin_example %}
+plugin: kong-inc/opentelemetry
+name: opentelemetry
+config:
+  traces_endpoint: "http://<opentelemetry-backend>:4318/v1/traces"
+  logs_endpoint: "http://<opentelemetry-backend>:4318/v1/logs"
+  resource_attributes:
+    service.name: kong-dev
+targets:
+  - service
+  - route
+  - consumer
+  - global
+formats:
+  - curl
+  - konnect
+  - yaml
+  - kubernetes
+{% endplugin_example %}
+{% endif_version %}
+<!--vale on-->
+
+## More information
+
+* [Troubleshooting the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#troubleshooting)
+* [Set up Dynatrace with OpenTelemetry](/hub/kong-inc/opentelemetry/how-to/dynatrace/)
+* [How logging works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#logging)
+* [How tracing works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#tracing)
diff --git a/app/_hub/kong-inc/opentelemetry/how-to/_spans.md b/app/_hub/kong-inc/opentelemetry/how-to/_spans.md
@@ -0,0 +1,46 @@
+---
+title: Customize OpenTelemetry spans as a developer
+nav_title: Customize OpenTelemetry spans as a developer
+---
+
+The OpenTelemetry plugin is built on top of the {{site.base_gateway}} tracing PDK.
+
+It's possible to customize the spans and add your own spans through the universal tracing PDK.
+
+## Create a custom span
+
+The following is an example for adding a custom span using {{site.base_gateway}}'s serverless plugin:
+
+1. Create a file named `custom-span.lua` with the following content:
+
+    ```lua
+      -- Modify the root span
+      local root_span = kong.tracing.active_span()
+      root_span:set_attribute("custom.attribute", "custom value")
+
+      -- Create a custom span
+      local span = kong.tracing.start_span("custom-span")
+
+      -- Append attributes
+      span:set_attribute("custom.attribute", "custom value")
+
+      -- Close the span
+      span:finish()
+    ```
+
+2. Apply the Lua code using the `post-function` plugin using a cURL file upload:
+
+    ```bash
+    curl -i -X POST http://localhost:8001/plugins \
+      -F "name=post-function" \
+      -F "config.access[1][email protected]"
+
+    HTTP/1.1 201 Created
+    ...
+    ```
+
+## More information
+
+* [Troubleshooting the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#troubleshooting)
+* [How logging works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#logging)
+* [How tracing works in the OpenTelemetry plugin](/hub/kong-inc/opentelemetry/#tracing)