New Blog Post: Collecting OpenTelemetry-compliant Java logs from files

[cyrille-leclerc]

This post presents how to use the new experimental-otlp/stdout export format introduced by OpenTelemetry Java Instrumentation (open-telemetry/opentelemetry-java#6675) to export application logs to files with full OpenTelemetry contextualization and ingest them with the OTel Collector File Log receiver and OTLP/JSON connector.

In this post, we explain how to configure the OTel Java instrumentation agent and the OTel Collector, including when deploying on K8s, to implement the end to end file based logs ingestion pipeline.

Blog Post Issue: #5606
SIG: java and specifications
Sponsor:

Preview: https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-otel-compliant-java-logs-from-files/

[zeitlinger]

@cyrille-leclerc https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v2.10.0 was just released - I'll update the executable example in grafana/docker-otel-lgtm#145 now

[svrnm]

@cyrille-leclerc can you raise an issue for this PR following our guidelines? Thanks

[cyrille-leclerc]

@cyrille-leclerc can you raise an issue for this PR following our guidelines? Thanks

Thanks @svrnm. Please find the blog post issue:

• New Blog Post: Collecting file-based Java logs with OpenTelemetry #5606

[svrnm]

the sample application you are using, is this available somewhere? If so, can you point to it? It's always good if people can reproduce your tutorial 1:1 without the need for them to find an application first

[cyrille-leclerc]

@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.

[zeitlinger]

yes, the demo app is at https://github.com/grafana/docker-otel-lgtm/tree/main/examples/java/json-logging-otlp - which is already mentioned twice

[svrnm]

I like that you provide the link to the otelbin source of that image, but let's figure out a way to visualize this better.

[svrnm]

Suggested change

	
	Source: [https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a](https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a)
	{{< figure class="figure" src="otel-collector-otlpjson-pipeline.png" attr="View OTel Collector pipeline with OTelBin" attrlink="https://www.otelbin.io/s/69739d790cf279c203fc8efc86ad1a876a2fc01a" >}}

What I meant is to have something that makes it clearer that the Source: (or View ...) belongs to the image, since the spacing is not right, similar to what bootstrap provides for figures. We have a shortcode for that via docsy, it's not exactly what I want, but we can take a look into that separately, cc @chalin

[chalin]

Has this been addressed to your satisfaction @svrnm? See, https://deploy-preview-5600--opentelemetry.netlify.app/blog/2024/collecting-file-based-java-logs-with-opentelemetry/#configure-the-opentelemetry-collector-to-ingest-the-otlpjson-logs.

[cyrille-leclerc]

@zeitlinger do we have a simplified demo or we use https://github.com/cyrille-leclerc/my-shopping-cart/? Note that I reverted my-shopping-cart to the otlp exporter so I would have to spend a few cycles to setup a demo.

[tiffany76]

Just a few copyedit suggestions. Thanks!

[chalin]

Why is this linking to a spec PR?
Is this PR really the best place for folks to comment?
If so, that's ok, but then the link text should explain more clearly what the link target is.

[cyrille-leclerc]

We have updated referring to the PR of the OTEP and to the Slack channel

[chalin]

/fix:refcache

[opentelemetrybot]

You triggered fix:refcache action run at https://github.com/open-telemetry/opentelemetry.io/actions/runs/11931927870

[opentelemetrybot]

fix:refcache was successful.

IMPORTANT: (RE-)RUN /fix:all to ensure that there are no remaining check issues.

[tiffany76]

LGTM, subject to @chalin's comments.

[cyrille-leclerc]

@svrnm FYI @zeitlinger and I think we are good to go. We reviewed and integrated all the feedback.
Can you please tell us what are the next steps?

Thanks again for your thorough review

[svrnm]

@cyrille-leclerc #5684 is blocking us right now, I will take another look when it is resolved and if all things look good we can merge it

[svrnm]

Suggested change

	date:
	date: 2024-12-02

please keep this open until we have set the exact date.

I would like to wait for @chalin to re-review and since this week most people are out due to Thanksgiving in the US I would argue that publishing next week is better anyways