Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pulling refs/heads/main into main #1617

Merged
merged 11 commits into from
Nov 26, 2024
4 changes: 3 additions & 1 deletion gdi/integrations-list.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ Supported integrations in Splunk Observability Cloud
Infrastructure monitoring <get-data-in/compute/compute>
APM instrumentation <get-data-in/application/application>
RUM instrumentation <get-data-in/rum/rum-instrumentation>
OpenTelemetry receivers <opentelemetry/components/a-components-receivers>
OpenTelemetry: Receivers <opentelemetry/components/a-components-receivers>
OpenTelemetry: Other ingestion methods <opentelemetry/otel-other/otel-other-landing>
Applications: Caches and memory TOGGLE <caches-memory>
Applications: Cloud platforms TOGGLE <cloud>
Applications: Cloudfoundry <monitors-cloudfoundry/cloudfoundry-firehose-nozzle>
Expand Down Expand Up @@ -130,6 +131,7 @@ You can monitor your applications and services with the Collector and the follow
You can also send data to Splunk Observability Cloud with OpenTelemetry with the following options:

* :ref:`prometheus-generic`
* :ref:`telegraf-generic`

.. raw:: html

Expand Down
2 changes: 0 additions & 2 deletions gdi/monitoring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,9 @@ Configure application exporters and receivers for monitoring

monitors-monitoring/cadvisor
monitors-monitoring/cgroups
Istio <get-data-in/application/istio/istio>
monitors-monitoring/jaeger-grpc
monitors-monitoring/nagios
monitors-monitoring/signalfx-forwarder
opentelemetry/components/splunk-apm-exporter
monitors-monitoring/win_perf_counters

These application receivers gather metrics from their associated monitoring applications and the hosts the applications are running on.
Expand Down
6 changes: 2 additions & 4 deletions gdi/monitors-databases/exec-input.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,9 @@ Exec Input (deprecated)

The Exec Input monitor is now deprecated and will reach of End of Support on February 3, 2025. During this period only critical security and bug fixes are provided. When End of Support is reached, the monitor will be removed and no longer be supported, and you won't be able to use it to send data to Splunk Observability Cloud.

To monitor your system with Telegraf Exec you can use native OpenTelemetry instead. See more at :new-page:`OpenTelemetry Output Plugin <https://github.com/influxdata/telegraf/blob/master/plugins/outputs/opentelemetry/README.md>` in GitHub.
To monitor your system with Telegraf Exec you can use native OpenTelemetry instead. See :ref:`telegraf-generic` to learn how.

The Splunk Distribution of the OpenTelemetry Collector uses the Smart Agent receiver with the
Exec Input monitor type, an embedded form of the Telegraf Exec plugin,
to receive metrics or logs from exec files.
The Splunk Distribution of the OpenTelemetry Collector uses the Smart Agent receiver with the Exec Input monitor type, an embedded form of the Telegraf Exec plugin, to receive metrics or logs from exec files.

Benefits
--------
Expand Down
21 changes: 21 additions & 0 deletions gdi/opentelemetry/otel-other/otel-other-landing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
.. _otel-other-landing:

******************************************
Other OpenTelemetry ingestion methods
******************************************

.. meta::
:description: Learn about other options to send your data with OpenTelemetry to Splunk Observability Cloud OpenTelemetry Collector.

.. toctree::
:maxdepth: 4
:titlesonly:
:hidden:

prometheus-generic
telegraf

On top of the available native :ref:`OpenTelemetry receivers <otel-components-receivers>`, you can also send data to Splunk Observability Cloud with OpenTelemetry with the following options:

* :ref:`prometheus-generic`
* :ref:`telegraf-generic`
123 changes: 123 additions & 0 deletions gdi/opentelemetry/otel-other/telegraf.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
.. _telegraf:
.. _telegraf-generic:

Monitor services with Telegraf and OpenTelemetry
========================================================

.. meta::
:description: Use this Splunk Observability Cloud integration for the Telegraf monitor. See benefits, install, configuration, and metrics.

To monitor your service with Telegraf using native OpenTelemetry in Splunk Observability Cloud, install the service's Telegraf plugin then push metrics to the Splunk Opentelemetry Collector
via OTLP.

.. note:: This setup is designed for a Linux Ubuntu OS but should be replicable on any machines running Linux OS with Debian flavor. These instructions might not work on other OS (MacOS/Windows).

Benefits
--------

.. include:: /_includes/benefits.rst

Configuration
----------------------------------

Follow these steps to scrape Telegraf metrics with the OTel Collector:

1. Install Telegraf
2. Set up your service's Telegraf Input plugin
3. Set up the Telegraf OpenTelemetry Output plugin
4. Configure the OpenTelemetry Collector

1. Install Telegraf
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Run the following commands to install Telegraf from the InfluxData repository:

.. code:: shell

curl --silent --location -O \ https://repos.influxdata.com/influxdata-archive.key \ && echo "943666881a1b8d9b849b74caebf02d3465d6beb716510d86a39f6c8e8dac7515 influxdata-archive.key" \
| sha256sum -c - && cat influxdata-archive.key \
| gpg --dearmor \
| sudo tee /etc/apt/trusted.gpg.d/influxdata-archive.gpg > /dev/null \
&& echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' \
| sudo tee /etc/apt/sources.list.d/influxdata.list
sudo apt-get update && sudo apt-get install telegraf

2. Set up your service's Telegraf Input plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Next, install the Telegraf Input plugin for the service you want to monitor. Available plugins include Chrony, Consul, Docker, Elasticsearch, Fluentd, GitHub, Jenkins, RabbitMQ or SQL. Find a complete list of Input plugins at :new-page:`Telegraf Input plugins <https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec>` in GitHub.

For example, if you want to monitor execute commands on every interval and parse metrics from their output with the exec input plugin, use a setup like:

.. code::

# Read metrics from one or more commands that can output to stdout
[[inputs.exec]]

## Commands array
commands = ["sh /testfolder/testscript.sh"]
timeout = "30s"
data_format = "influx"

## Environment variables
## Array of "key=value" pairs to pass as environment variables
## e.g. "KEY=value", "USERNAME=John Doe",
## "LD_LIBRARY_PATH=/opt/custom/lib64:/usr/local/libs"
# environment = []

## Measurement name suffix
## Used for separating different commands
# name_suffix = ""

## Ignore Error Code
## If set to true, a non-zero error code in not considered an error and the
## plugin will continue to parse the output.
# ignore_error = false

3. Set up the Telegraf OpenTelemetry Output plugin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Next, add the OTel Output plugin to your Telegraf configuration file:

.. code::

# Send OpenTelemetry metrics over gRPC
[[outputs.opentelemetry]]

The config file usually resides on the ./etc/telegraf/telegraf.d directory.

For detailed information see Telegraf's :new-page:`OpenTelemetry Output plugin <https://github.com/influxdata/telegraf/blob/master/plugins/outputs/opentelemetry/README.md>` documentation in GitHub.

4. Configure the OpenTelemetry Collector
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Add the following configuration to the OTel Collector to retrieve metrics from the Telegraf installation:

.. code:: yaml

receivers:
otlp:
protocols:
http:
grpc:
signalfx:

exporters:
signalfx:
access_token: "SPLUNK_TOKEN"
realm: "us0"

service:
pipelines:
metrics:
receivers: [otlp]
exporters: [signalfx]
metrics/internal:
receivers: [signalfx]
processors:
exporters: [signalfx]

Troubleshooting
---------------

.. include:: /_includes/troubleshooting-components.rst
13 changes: 5 additions & 8 deletions gdi/prometheus.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,21 @@ Configure application receivers with Prometheus metrics
:maxdepth: 4
:hidden:

monitors-prometheus/prometheus-generic
opentelemetry/components/prometheus-receiver
monitors-prometheus/prometheus-exporter
monitors-prometheus/prometheus-go
monitors-prometheus/prometheus-nginx-ingress
monitors-prometheus/prometheus-nginx-vts
monitors-prometheus/prometheus-node
monitors-prometheus/prometheus-velero

You can use the OpenTelementry native Prometheus receiver to gather metrics from any software, tool or service that exposes metrics in the Prometheus format, such as Ansible Tower (AWX), CockroachDB, Docker Daemon, Doorman, Etcd, Flink, Grafana, PATROL, Traefik, or Zipkin. See more at :ref:`prometheus-receiver`.
Splunk Observability Cloud offers the following OTel solutions to monitor applications and services that expose metrics in the Prometheus format:

Splunk Observability Cloud offers the following solutions:
* :ref:`prometheus-receiver`. You can use the OpenTelementry native Prometheus receiver to gather metrics from any software, tool or service that exposes metrics in the Prometheus format, such as Ansible Tower (AWX), CockroachDB, Docker Daemon, Doorman, Etcd, Flink, Grafana, PATROL, Traefik, or Zipkin.
* :ref:`prometheus-generic`.

* :ref:`prometheus-generic`
* :ref:`prometheus-receiver`
* See a complete list of third-party applications compatible with Prometheus in :new-page:`Prometheus' official documentation <https://prometheus.io/docs/instrumenting/exporters/>`.
You can find the complete list of third-party applications compatible with Prometheus in :new-page:`Prometheus' official documentation <https://prometheus.io/docs/instrumenting/exporters/>`.

See also:
The following Smart Agent monitors are also available:

* :ref:`prometheus-exporter`
* :ref:`prometheus-go`
Expand Down