Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New component: Exporter for OpenSearch #14

Open
1 of 2 tasks
MaxKsyunz opened this issue Jun 8, 2023 · 0 comments
Open
1 of 2 tasks

New component: Exporter for OpenSearch #14

MaxKsyunz opened this issue Jun 8, 2023 · 0 comments

Comments

@MaxKsyunz
Copy link

MaxKsyunz commented Jun 8, 2023

The purpose and use-cases of the new component

An exporter for OpenSearch that sends signals directly to an OpenSearch cluster in a schema that is understood by the observability plugin.

It assumes that destination indices have the same mapping as data prepper.

Example configuration for the component

The exporter will have the following configuration options:

  • endpoints -- List of OpenSearch URLs. If endpoints is missing, the OPENSEARCH_URL environment variable will be used.
  • num_workers (optional) -- Number of workers publishing bulk requests concurrently.
  • traces_index -- name of the index to publish traces to
  • flush -- Event bulk buffer flush settings
    • bytes (default=5242880): Write buffer flush limit.
    • interval (default=30s): Write buffer time limit.
  • retry: Event retry settings
    • enabled (default=true): Enable/Disable event retry on error. Retry support is enabled by default.
    • max_requests (default=3): Number of HTTP request retries.
    • initial_interval (default=100ms): Initial waiting time if a HTTP request failed.
    • max_interval (default=1m): Max waiting time if a HTTP request failed.

Telemetry data types supported

The exporter will support traces.

Is this a vendor-specific component?

  • This is a vendor-specific component
  • If this is a vendor-specific component, I am proposing to contribute this as a representative of the vendor.

Sponsor (optional)

No response

Additional context

Metrics, logs and simple schema for observability support will be handled as separate PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant