Introduce a new flag to explicitly permit legacy monitoring #16586

andsel · 2024-10-18T14:46:07Z

Release notes

Introduce a new flag setting xpack.monitoring.allow_legacy_collection which eventually enable the legacy monitoring collector.

What does this PR do?

Update the method to test if monitoring is enabled so that consider also xpack.monitoring.allow_legacy_collection to determine if monitoring.* settings are valid or not.
By default it's false, the user has to intentionally enable it to continue to use the legacy monitoring settings.

Why is it important/What is the impact to the user?

Permit to the user to force the usage of X-Pack legacy internal collector

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
~~[ ] I have made corresponding change to the default configuration files (and/or docker env variables)~~
I have added tests that prove my fix is effective or that my feature works

Author's Checklist

[ ]

How to test this PR locally

Run ES and KB locally with:

curl -fsSL https://elastic.co/start-local | sh

from the console record the password for elastic user.
configure Logstash internal monitoring (in config/logstash.yml), like:

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: elastic
xpack.monitoring.elasticsearch.password:s3cr3t
xpack.monitoring.elasticsearch.hosts: ["http://localhost:9200"]

verify no monitoring data is present in Kibana
stop Logstash
override the legacy collection enabling explicitly in config/logstash.yml with:

xpack.monitoring.allow_legacy_collection: true

verify that Kibana can show monitoring data

Related issues

Closes Move Internal Collection stack monitoring behind a feature flag #16481

Use cases

As a developer I want to definitively drop the internal legacy collector in favor of ElasticAgent

Logs

Logs without collection override

% bin/logstash
Using system java: /Users/andrea/.sdkman/candidates/java/current/bin/java
Sending Logstash logs to /Users/andrea/workspace/logstash_andsel/logs which is now configured via log4j2.properties
[2024-10-21T10:12:39,642][INFO ][logstash.runner          ] Log4j configuration path used is: /Users/andrea/workspace/logstash_andsel/config/log4j2.properties
[2024-10-21T10:12:39,645][WARN ][logstash.runner          ] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2024-10-21T10:12:39,645][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"9.0.0", "jruby.version"=>"jruby 9.4.8.0 (3.1.4) 2024-07-02 4d41e55a67 OpenJDK 64-Bit Server VM 21.0.4+7-LTS on 21.0.4+7-LTS +indy +jit [arm64-darwin]"}
[2024-10-21T10:12:39,646][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[2024-10-21T10:12:39,647][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
[2024-10-21T10:12:39,647][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
[2024-10-21T10:12:39,655][INFO ][logstash.configmanagement.bootstrapcheck] Using Elasticsearch as config store {:pipeline_id=>["main", "apache_logs"], :poll_interval=>"TimeValue{duration=5, timeUnit=SECONDS}ns"}
[2024-10-21T10:12:39,811][INFO ][logstash.configmanagement.elasticsearchsource] Configuration Management License OK
[2024-10-21T10:12:40,003][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-10-21T10:12:40,006][INFO ][logstash.configmanagement.elasticsearchsource] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@localhost:9200/]}}
[2024-10-21T10:12:40,017][WARN ][logstash.configmanagement.elasticsearchsource] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@localhost:9200/"}
[2024-10-21T10:12:40,017][INFO ][logstash.configmanagement.elasticsearchsource] Elasticsearch version determined (8.15.3) {:es_version=>8}
[2024-10-21T10:12:40,017][WARN ][logstash.configmanagement.elasticsearchsource] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2024-10-21T10:12:40,111][INFO ][org.reflections.Reflections] Reflections took 53 ms to scan 1 urls, producing 149 keys and 522 values
[2024-10-21T10:12:40,217][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2024-10-21T10:12:40,222][WARN ][logstash.javapipeline    ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2024-10-21T10:12:40,227][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["central pipeline management"], :thread=>"#<Thread:0x3de7334d /Users/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/java_pipeline.rb:139 run>"}
[2024-10-21T10:12:40,406][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.18}
[2024-10-21T10:12:40,415][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[2024-10-21T10:12:40,423][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
^C[2024-10-21T10:12:54,390][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2024-10-21T10:12:54,477][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2024-10-21T10:12:55,411][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:main}
[2024-10-21T10:12:55,419][INFO ][logstash.runner          ] Logstash shut down.

Logs with collection override

% bin/logstash
Using system java: /Users/andrea/.sdkman/candidates/java/current/bin/java
Sending Logstash logs to /Users/andrea/workspace/logstash_andsel/logs which is now configured via log4j2.properties
[2024-10-21T10:13:19,015][INFO ][logstash.runner          ] Log4j configuration path used is: /Users/andrea/workspace/logstash_andsel/config/log4j2.properties
[2024-10-21T10:13:19,017][WARN ][logstash.runner          ] The use of JAVA_HOME has been deprecated. Logstash 8.0 and later ignores JAVA_HOME and uses the bundled JDK. Running Logstash with the bundled JDK is recommended. The bundled JDK has been verified to work with each specific version of Logstash, and generally provides best performance and reliability. If you have compelling reasons for using your own JDK (organizational-specific compliance requirements, for example), you can configure LS_JAVA_HOME to use that version instead.
[2024-10-21T10:13:19,017][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"9.0.0", "jruby.version"=>"jruby 9.4.8.0 (3.1.4) 2024-07-02 4d41e55a67 OpenJDK 64-Bit Server VM 21.0.4+7-LTS on 21.0.4+7-LTS +indy +jit [arm64-darwin]"}
[2024-10-21T10:13:19,018][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true, -Dlogstash.jackson.stream-read-constraints.max-string-length=200000000, -Dlogstash.jackson.stream-read-constraints.max-number-length=10000, -Djruby.regexp.interruptible=true, -Djdk.io.File.enableADS=true, --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED, --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED, --add-opens=java.base/java.security=ALL-UNNAMED, --add-opens=java.base/java.io=ALL-UNNAMED, --add-opens=java.base/java.nio.channels=ALL-UNNAMED, --add-opens=java.base/sun.nio.ch=ALL-UNNAMED, --add-opens=java.management/sun.management=ALL-UNNAMED, -Dio.netty.allocator.maxOrder=11]
[2024-10-21T10:13:19,019][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-string-length` configured to `200000000`
[2024-10-21T10:13:19,019][INFO ][logstash.runner          ] Jackson default value override `logstash.jackson.stream-read-constraints.max-number-length` configured to `10000`
[2024-10-21T10:13:19,026][INFO ][logstash.configmanagement.bootstrapcheck] Using Elasticsearch as config store {:pipeline_id=>["main", "apache_logs"], :poll_interval=>"TimeValue{duration=5, timeUnit=SECONDS}ns"}
[2024-10-21T10:13:19,187][INFO ][logstash.configmanagement.elasticsearchsource] Configuration Management License OK
[2024-10-21T10:13:19,427][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
[2024-10-21T10:13:19,427][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
[2024-10-21T10:13:19,445][INFO ][logstash.configmanagement.elasticsearchsource] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@localhost:9200/]}}
[2024-10-21T10:13:19,464][WARN ][logstash.configmanagement.elasticsearchsource] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@localhost:9200/"}
[2024-10-21T10:13:19,464][INFO ][logstash.configmanagement.elasticsearchsource] Elasticsearch version determined (8.15.3) {:es_version=>8}
[2024-10-21T10:13:19,464][WARN ][logstash.configmanagement.elasticsearchsource] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2024-10-21T10:13:19,466][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600, :ssl_enabled=>false}
[2024-10-21T10:13:19,588][INFO ][org.reflections.Reflections] Reflections took 69 ms to scan 1 urls, producing 149 keys and 522 values
[2024-10-21T10:13:19,640][INFO ][logstash.javapipeline    ] Pipeline `.monitoring-logstash` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2024-10-21T10:13:19,649][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["http://localhost:9200"]}
[2024-10-21T10:13:19,652][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@localhost:9200/]}}
[2024-10-21T10:13:19,664][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@localhost:9200/"}
[2024-10-21T10:13:19,664][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch version determined (8.15.3) {:es_version=>8}
[2024-10-21T10:13:19,664][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>8}
[2024-10-21T10:13:19,671][WARN ][logstash.javapipeline    ][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2024-10-21T10:13:19,675][INFO ][logstash.javapipeline    ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x6075fcff /Users/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/java_pipeline.rb:139 run>"}
[2024-10-21T10:13:19,676][INFO ][logstash.javapipeline    ] Pipeline `main` is configured with `pipeline.ecs_compatibility: v8` setting. All plugins in this pipeline will default to `ecs_compatibility => v8` unless explicitly configured otherwise.
[2024-10-21T10:13:19,678][WARN ][logstash.javapipeline    ][main] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
[2024-10-21T10:13:19,679][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["central pipeline management"], :thread=>"#<Thread:0x5f62301d /Users/andrea/workspace/logstash_andsel/logstash-core/lib/logstash/java_pipeline.rb:139 run>"}
[2024-10-21T10:13:19,866][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.19}
[2024-10-21T10:13:19,866][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>0.19}
[2024-10-21T10:13:19,875][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
[2024-10-21T10:13:19,896][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[2024-10-21T10:13:19,907][INFO ][logstash.agent           ] Pipelines running {:count=>2, :running_pipelines=>[:".monitoring-logstash", :main], :non_running_pipelines=>[]}
^C[2024-10-21T10:14:36,525][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2024-10-21T10:14:36,616][INFO ][logstash.javapipeline    ][main] Pipeline terminated {"pipeline.id"=>"main"}
[2024-10-21T10:14:37,544][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:main}
[2024-10-21T10:14:37,968][INFO ][logstash.javapipeline    ][.monitoring-logstash] Pipeline terminated {"pipeline.id"=>".monitoring-logstash"}
[2024-10-21T10:14:38,553][INFO ][logstash.pipelinesregistry] Removed pipeline from registry successfully {:pipeline_id=>:".monitoring-logstash"}
[2024-10-21T10:14:38,562][INFO ][logstash.runner          ] Logstash shut down.

kaisecheng

Besides the suggestion of renaming the flag, I would suggest to add allow.legacy.monitoring to env2yaml.go. The main reason is to ease the transition for docker users as they can setup monitoring by just passing the env variable today.

docs/static/settings-file.asciidoc

config/logstash.yml

kaisecheng

please add the flag to benchmark buildkite 🙏

When I use the xpack.* to enable monitoring in old way, Logstash does not give any hints/ logs that it is missing the flag. I think we need to print logs to guide user.

Let @karenzone to have a look on the words, otherwise, LGTM

docs/static/monitoring/monitoring-internal-legacy.asciidoc

config/logstash.yml

x-pack/spec/monitoring/pipeline_register_hook_spec.rb

kaisecheng · 2024-10-22T11:34:42Z

One more missing in default_metadata.rb for curl localhost:9600/

andsel · 2024-10-22T12:31:30Z

Thanks @kaisecheng for your review, added the allow.legacy.monitoring flag to BK pipeline and now logs a warn when legacy monitoring is enabled and not allowed.

Example of line logged:

[2024-10-22T14:27:17,977][WARN ][logstash.monitoringextension.pipelineregisterhook] Legacy internal monitoring is enabled (xpack.monitoring.enabled or monitoring.enabled is true) but not allowed. Please check your configuration and eventually set allow.legacy.monitoring

kaisecheng

One more missing in default_metadata.rb for curl localhost:9600/

☝️ Almost there

x-pack/lib/monitoring/monitoring.rb

andsel · 2024-10-22T13:41:16Z

@kaisecheng

☝️ Almost there

was fixed in https://github.com/elastic/logstash/pull/16586/files#diff-a0255510e1f939e6ad6e719d7db5ec42a9f0e886171561b1084da93693a44256R6 if I'm not wrong

kaisecheng · 2024-10-22T13:52:55Z

@andsel https://github.com/elastic/logstash/blob/main/logstash-core/lib/logstash/api/commands/default_metadata.rb#L73-L76
It is about logstash API. When enabled_xpack_monitoring? is enabled, monitoring related info will return. I think it needs to add && LogStash::SETTINGS.get("allow.legacy.monitoring")

kaisecheng · 2024-10-22T13:54:45Z

@andsel btw, I have triggered exhausted test for this pr, but got red

kaisecheng

LGTM :)

karenzone

Does this apply only to internal collection? If so, we should call out internal collection specifically. Technically, any collection method we've used in the past would fall under the "legacy" umbrella, and we might need more precision and granularity than would be provided by labeling this one implementation as legacy.
Please LMKWYT.

andsel · 2024-11-04T08:07:49Z

@karenzone yes it's related just to the internal collector. Do you thing we should switch naming from allow.legacy.monitoring to allow.internal.legacy.monitoring ?

robbavey

I think we should consider renaming the setting to live under xpack.monitoring WDYT?

.buildkite/scripts/benchmark/config/logstash.yml

…ally enable the legacy monitoring collector

…toring.

…monitoring is overridden

Co-authored-by: kaisecheng <[email protected]>

… but not allowed.

Co-authored-by: kaisecheng <[email protected]>

…ollection' and moved the registering of such setting from core to xpack

docs/static/monitoring/monitoring-internal-legacy.asciidoc

Co-authored-by: Karen Metts <[email protected]>

elastic-sonarqube · 2024-11-18T08:16:40Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

elasticmachine · 2024-11-18T08:29:21Z

💚 Build Succeeded

Buildkite Build
Commit: a5a2028

History

💚 Build #1853 succeeded dcb0c25
💛 Build #1851 was flaky c0e0aba
💚 Build #1850 succeeded 20b2db2
💔 Build #1849 failed e8a2d72
💚 Build #1747 succeeded 3d5ab00
💚 Build #1746 succeeded 8abc793

cc @andsel

karenzone

LGTM

In elastic#16586 the module include was removed. This causes failures in the script when the module is referenced. This commit re-enables the include for the yaml module.

In #16586 the module include was removed. This causes failures in the script when the module is referenced. This commit re-enables the include for the yaml module.

andsel self-assigned this Oct 18, 2024

andsel marked this pull request as draft October 18, 2024 14:46

andsel changed the title ~~Introduce a new flag setting 'legacy.monitoring.enabled' which eventu…~~ Introduce a new flag to explicitly permit legacy monitoring Oct 18, 2024

andsel linked an issue Oct 18, 2024 that may be closed by this pull request

Move Internal Collection stack monitoring behind a feature flag #16481

Closed

jsvd added the status:work-in-progress label Oct 18, 2024

andsel added the enhancement label Oct 21, 2024

andsel marked this pull request as ready for review October 21, 2024 08:42

jsvd added status:needs-triage and removed status:work-in-progress labels Oct 21, 2024

kaisecheng reviewed Oct 22, 2024

View reviewed changes

docs/static/settings-file.asciidoc Outdated Show resolved Hide resolved

config/logstash.yml Outdated Show resolved Hide resolved

config/logstash.yml Outdated Show resolved Hide resolved

andsel requested a review from kaisecheng October 22, 2024 09:20

kaisecheng requested changes Oct 22, 2024

View reviewed changes

docs/static/monitoring/monitoring-internal-legacy.asciidoc Outdated Show resolved Hide resolved

config/logstash.yml Outdated Show resolved Hide resolved

x-pack/spec/monitoring/pipeline_register_hook_spec.rb Outdated Show resolved Hide resolved

jsvd added status:changes-requested and removed status:needs-triage labels Oct 22, 2024

andsel requested review from kaisecheng and karenzone October 22, 2024 13:03

kaisecheng reviewed Oct 22, 2024

View reviewed changes

x-pack/lib/monitoring/monitoring.rb Outdated Show resolved Hide resolved

kaisecheng approved these changes Oct 23, 2024

View reviewed changes

jsvd added status:approved and removed status:changes-requested labels Oct 23, 2024

karenzone reviewed Oct 31, 2024

View reviewed changes

robbavey requested changes Nov 5, 2024

View reviewed changes

.buildkite/scripts/benchmark/config/logstash.yml Outdated Show resolved Hide resolved

andsel requested review from robbavey and karenzone November 13, 2024 10:14

andsel and others added 17 commits November 14, 2024 10:49

Introduce a new flag setting 'legacy.monitoring.enabled' which eventu…

3d00722

…ally enable the legacy monitoring collector

Removed commented line

e9677e0

[Docs] Added note to call the user that want to force the legacy moni…

ad6c4d2

…toring.

[test] Added test to cover monitoring to be enabled only when legacy.…

a5745b3

…monitoring is overridden

Apply suggestions from code review

ba56ec5

Co-authored-by: kaisecheng <[email protected]>

Renamed 'legacy.monitoring.enabled' to 'allow.legacy.monitoring'

d4fffb1

Exposed 'allow.legacy.monitoring' from Docker image

1bc7542

Apply suggestions from code review

cf827c8

Co-authored-by: kaisecheng <[email protected]>

Updated benchmark BK pipeline to explicitly allow legacy collection

f2bcee5

Logs a warning message when detects that legacy monitoring is enabled…

367844a

… but not allowed.

Update x-pack/lib/monitoring/monitoring.rb

ae993ba

Co-authored-by: kaisecheng <[email protected]>

Fixed sunthax error

7af86bd

Added allow legacy flag to expose montoring section in API

9d95549

Re-added yaml parsing lib that made the go build phase to fail

4838791

Renamed 'allow.legacy.monitoring' to 'xpack.monitoring.allow_legacy_c…

08b7c8a

…ollection' and moved the registering of such setting from core to xpack

Updated consistently the docs

1d16b13

Fixed filtering rule on root 'xpack.monitoring.'

c0e0aba

andsel force-pushed the fix/add_feature_flag_to_forcibly_enable_legacy_monitoring_collector branch from 20b2db2 to c0e0aba Compare November 14, 2024 09:49

andsel added 2 commits November 14, 2024 15:58

Missed to update flag name in BK benchmark pipeline

2eda740

Minor, removed unused code

dcb0c25

karenzone reviewed Nov 15, 2024

View reviewed changes

docs/static/monitoring/monitoring-internal-legacy.asciidoc Outdated Show resolved Hide resolved

Re-phrased the note on the intro of new flag

a5a2028

Co-authored-by: Karen Metts <[email protected]>

andsel requested a review from karenzone November 18, 2024 08:30

karenzone approved these changes Nov 18, 2024

View reviewed changes

andsel merged commit d4fb06e into elastic:main Nov 19, 2024
7 checks passed

donoghuc mentioned this pull request Nov 20, 2024

Replace removed yaml module #16703

Merged

donoghuc added a commit that referenced this pull request Nov 20, 2024

Replace removed yaml module (#16703)

7b3d23b

In #16586 the module include was removed. This causes failures in the script when the module is referenced. This commit re-enables the include for the yaml module.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce a new flag to explicitly permit legacy monitoring #16586

Introduce a new flag to explicitly permit legacy monitoring #16586

andsel commented Oct 18, 2024 •

edited

Loading

kaisecheng left a comment

kaisecheng left a comment

kaisecheng commented Oct 22, 2024

andsel commented Oct 22, 2024

kaisecheng left a comment

andsel commented Oct 22, 2024

kaisecheng commented Oct 22, 2024

kaisecheng commented Oct 22, 2024

kaisecheng left a comment

karenzone left a comment

andsel commented Nov 4, 2024

robbavey left a comment

elastic-sonarqube bot commented Nov 18, 2024

elasticmachine commented Nov 18, 2024

karenzone left a comment

Introduce a new flag to explicitly permit legacy monitoring #16586

Introduce a new flag to explicitly permit legacy monitoring #16586

Conversation

andsel commented Oct 18, 2024 • edited Loading

Release notes

What does this PR do?

Why is it important/What is the impact to the user?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Logs

Logs without collection override

Logs with collection override

kaisecheng left a comment

Choose a reason for hiding this comment

kaisecheng left a comment

Choose a reason for hiding this comment

kaisecheng commented Oct 22, 2024

andsel commented Oct 22, 2024

kaisecheng left a comment

Choose a reason for hiding this comment

andsel commented Oct 22, 2024

kaisecheng commented Oct 22, 2024

kaisecheng commented Oct 22, 2024

kaisecheng left a comment

Choose a reason for hiding this comment

karenzone left a comment

Choose a reason for hiding this comment

andsel commented Nov 4, 2024

robbavey left a comment

Choose a reason for hiding this comment

elastic-sonarqube bot commented Nov 18, 2024

Quality Gate passed

elasticmachine commented Nov 18, 2024

💚 Build Succeeded

History

karenzone left a comment

Choose a reason for hiding this comment

andsel commented Oct 18, 2024 •

edited

Loading