Semi structured json log parsing example

[pepov]

It would be nice to have an example to demonstrate how to parse semi-structured logs.

Let's consider the following log record:

  "message": [
    "I0104 15:47:24.388872   118 SyslogReporter.cpp:10] APP-METRIC {\"fields\":{\"drop\":0},\"measurement\":\"core.datafield\",\"tags\":{\"host\":\"blahblah-testing-9rtvp\",\"instance\":\"0\",\"source\":\"core.datafield\",\"type\":\"data\",\"worker\":\"0\"},\"time\":\"2024-01-04T15:47:24.000000000Z\"}"
],

We want to be able to parse this log message into a fully structured representation, typically into json. Let's create a test setup with a flow that solves this as a blog post or simply as an example in the docs:

logging-operator/pkg/sdk/logging/model/filter/parser.go

Line 203 in 4a926ef

Ideally the example should represent both the fluentd and syslog-ng solution.