diff --git a/gdi/integrations-list.rst b/gdi/integrations-list.rst index a89d4da75..13e2ac245 100644 --- a/gdi/integrations-list.rst +++ b/gdi/integrations-list.rst @@ -19,7 +19,8 @@ Supported integrations in Splunk Observability Cloud Infrastructure monitoring APM instrumentation RUM instrumentation - OpenTelemetry receivers + OpenTelemetry: Receivers + OpenTelemetry: Other ingestion methods Applications: Caches and memory TOGGLE Applications: Cloud platforms TOGGLE Applications: Cloudfoundry @@ -130,6 +131,7 @@ You can monitor your applications and services with the Collector and the follow You can also send data to Splunk Observability Cloud with OpenTelemetry with the following options: * :ref:`prometheus-generic` +* :ref:`telegraf-generic` .. raw:: html diff --git a/gdi/monitoring.rst b/gdi/monitoring.rst index b8483c842..8b8ef6c02 100644 --- a/gdi/monitoring.rst +++ b/gdi/monitoring.rst @@ -13,11 +13,9 @@ Configure application exporters and receivers for monitoring monitors-monitoring/cadvisor monitors-monitoring/cgroups - Istio monitors-monitoring/jaeger-grpc monitors-monitoring/nagios monitors-monitoring/signalfx-forwarder - opentelemetry/components/splunk-apm-exporter monitors-monitoring/win_perf_counters These application receivers gather metrics from their associated monitoring applications and the hosts the applications are running on. diff --git a/gdi/monitors-databases/exec-input.rst b/gdi/monitors-databases/exec-input.rst index c0a40b07b..3bbd292d9 100644 --- a/gdi/monitors-databases/exec-input.rst +++ b/gdi/monitors-databases/exec-input.rst @@ -10,11 +10,9 @@ Exec Input (deprecated) The Exec Input monitor is now deprecated and will reach of End of Support on February 3, 2025. During this period only critical security and bug fixes are provided. When End of Support is reached, the monitor will be removed and no longer be supported, and you won't be able to use it to send data to Splunk Observability Cloud. - To monitor your system with Telegraf Exec you can use native OpenTelemetry instead. See more at :new-page:`OpenTelemetry Output Plugin ` in GitHub. + To monitor your system with Telegraf Exec you can use native OpenTelemetry instead. See :ref:`telegraf-generic` to learn how. -The Splunk Distribution of the OpenTelemetry Collector uses the Smart Agent receiver with the -Exec Input monitor type, an embedded form of the Telegraf Exec plugin, -to receive metrics or logs from exec files. +The Splunk Distribution of the OpenTelemetry Collector uses the Smart Agent receiver with the Exec Input monitor type, an embedded form of the Telegraf Exec plugin, to receive metrics or logs from exec files. Benefits -------- diff --git a/gdi/opentelemetry/otel-other/otel-other-landing.rst b/gdi/opentelemetry/otel-other/otel-other-landing.rst new file mode 100644 index 000000000..0137a1661 --- /dev/null +++ b/gdi/opentelemetry/otel-other/otel-other-landing.rst @@ -0,0 +1,21 @@ +.. _otel-other-landing: + +****************************************** +Other OpenTelemetry ingestion methods +****************************************** + +.. meta:: + :description: Learn about other options to send your data with OpenTelemetry to Splunk Observability Cloud OpenTelemetry Collector. + +.. toctree:: + :maxdepth: 4 + :titlesonly: + :hidden: + + prometheus-generic + telegraf + +On top of the available native :ref:`OpenTelemetry receivers `, you can also send data to Splunk Observability Cloud with OpenTelemetry with the following options: + +* :ref:`prometheus-generic` +* :ref:`telegraf-generic` \ No newline at end of file diff --git a/gdi/monitors-prometheus/prometheus-generic.rst b/gdi/opentelemetry/otel-other/prometheus-generic.rst similarity index 100% rename from gdi/monitors-prometheus/prometheus-generic.rst rename to gdi/opentelemetry/otel-other/prometheus-generic.rst diff --git a/gdi/opentelemetry/otel-other/telegraf.rst b/gdi/opentelemetry/otel-other/telegraf.rst new file mode 100644 index 000000000..94fe139d3 --- /dev/null +++ b/gdi/opentelemetry/otel-other/telegraf.rst @@ -0,0 +1,123 @@ +.. _telegraf: +.. _telegraf-generic: + +Monitor services with Telegraf and OpenTelemetry +======================================================== + +.. meta:: + :description: Use this Splunk Observability Cloud integration for the Telegraf monitor. See benefits, install, configuration, and metrics. + +To monitor your service with Telegraf using native OpenTelemetry in Splunk Observability Cloud, install the service's Telegraf plugin then push metrics to the Splunk Opentelemetry Collector +via OTLP. + +.. note:: This setup is designed for a Linux Ubuntu OS but should be replicable on any machines running Linux OS with Debian flavor. These instructions might not work on other OS (MacOS/Windows). + +Benefits +-------- + +.. include:: /_includes/benefits.rst + +Configuration +---------------------------------- + +Follow these steps to scrape Telegraf metrics with the OTel Collector: + +1. Install Telegraf +2. Set up your service's Telegraf Input plugin +3. Set up the Telegraf OpenTelemetry Output plugin +4. Configure the OpenTelemetry Collector + +1. Install Telegraf +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Run the following commands to install Telegraf from the InfluxData repository: + +.. code:: shell + + curl --silent --location -O \ https://repos.influxdata.com/influxdata-archive.key \ && echo "943666881a1b8d9b849b74caebf02d3465d6beb716510d86a39f6c8e8dac7515 influxdata-archive.key" \ + | sha256sum -c - && cat influxdata-archive.key \ + | gpg --dearmor \ + | sudo tee /etc/apt/trusted.gpg.d/influxdata-archive.gpg > /dev/null \ + && echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' \ + | sudo tee /etc/apt/sources.list.d/influxdata.list + sudo apt-get update && sudo apt-get install telegraf + +2. Set up your service's Telegraf Input plugin +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Next, install the Telegraf Input plugin for the service you want to monitor. Available plugins include Chrony, Consul, Docker, Elasticsearch, Fluentd, GitHub, Jenkins, RabbitMQ or SQL. Find a complete list of Input plugins at :new-page:`Telegraf Input plugins ` in GitHub. + +For example, if you want to monitor execute commands on every interval and parse metrics from their output with the exec input plugin, use a setup like: + +.. code:: + + # Read metrics from one or more commands that can output to stdout + [[inputs.exec]] + + ## Commands array + commands = ["sh /testfolder/testscript.sh"] + timeout = "30s" + data_format = "influx" + + ## Environment variables + ## Array of "key=value" pairs to pass as environment variables + ## e.g. "KEY=value", "USERNAME=John Doe", + ## "LD_LIBRARY_PATH=/opt/custom/lib64:/usr/local/libs" + # environment = [] + + ## Measurement name suffix + ## Used for separating different commands + # name_suffix = "" + + ## Ignore Error Code + ## If set to true, a non-zero error code in not considered an error and the + ## plugin will continue to parse the output. + # ignore_error = false + +3. Set up the Telegraf OpenTelemetry Output plugin +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Next, add the OTel Output plugin to your Telegraf configuration file: + +.. code:: + + # Send OpenTelemetry metrics over gRPC + [[outputs.opentelemetry]] + +The config file usually resides on the ./etc/telegraf/telegraf.d directory. + +For detailed information see Telegraf's :new-page:`OpenTelemetry Output plugin ` documentation in GitHub. + +4. Configure the OpenTelemetry Collector +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add the following configuration to the OTel Collector to retrieve metrics from the Telegraf installation: + +.. code:: yaml + + receivers: + otlp: + protocols: + http: + grpc: + signalfx: + + exporters: + signalfx: + access_token: "SPLUNK_TOKEN" + realm: "us0" + + service: + pipelines: + metrics: + receivers: [otlp] + exporters: [signalfx] + metrics/internal: + receivers: [signalfx] + processors: + exporters: [signalfx] + +Troubleshooting +--------------- + +.. include:: /_includes/troubleshooting-components.rst diff --git a/gdi/prometheus.rst b/gdi/prometheus.rst index 63d18d9be..3cba12379 100644 --- a/gdi/prometheus.rst +++ b/gdi/prometheus.rst @@ -11,8 +11,6 @@ Configure application receivers with Prometheus metrics :maxdepth: 4 :hidden: - monitors-prometheus/prometheus-generic - opentelemetry/components/prometheus-receiver monitors-prometheus/prometheus-exporter monitors-prometheus/prometheus-go monitors-prometheus/prometheus-nginx-ingress @@ -20,15 +18,14 @@ Configure application receivers with Prometheus metrics monitors-prometheus/prometheus-node monitors-prometheus/prometheus-velero -You can use the OpenTelementry native Prometheus receiver to gather metrics from any software, tool or service that exposes metrics in the Prometheus format, such as Ansible Tower (AWX), CockroachDB, Docker Daemon, Doorman, Etcd, Flink, Grafana, PATROL, Traefik, or Zipkin. See more at :ref:`prometheus-receiver`. +Splunk Observability Cloud offers the following OTel solutions to monitor applications and services that expose metrics in the Prometheus format: -Splunk Observability Cloud offers the following solutions: +* :ref:`prometheus-receiver`. You can use the OpenTelementry native Prometheus receiver to gather metrics from any software, tool or service that exposes metrics in the Prometheus format, such as Ansible Tower (AWX), CockroachDB, Docker Daemon, Doorman, Etcd, Flink, Grafana, PATROL, Traefik, or Zipkin. +* :ref:`prometheus-generic`. -* :ref:`prometheus-generic` -* :ref:`prometheus-receiver` -* See a complete list of third-party applications compatible with Prometheus in :new-page:`Prometheus' official documentation `. +You can find the complete list of third-party applications compatible with Prometheus in :new-page:`Prometheus' official documentation `. -See also: +The following Smart Agent monitors are also available: * :ref:`prometheus-exporter` * :ref:`prometheus-go`