initial implementation #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mprimi
approved these changes
Jan 16, 2024
| shell: bash --noprofile --norc -x -eo pipefail {0} | ||
| run: | | ||
| cd go/src/github.com/nats-io/nats-server | ||
| go test -v --run='-' --bench 'MQTTEx' --benchtime=100x ./server | tee /tmp/current-bench-result.txt |
There was a problem hiding this comment.
Is this a test inside nats-server that relies on an external CLI tool (this repository)?
(maybe to avoid extra MQTT dependencies in server?)
There was a problem hiding this comment.
Exactly, see nats-io/nats-server#4821
But it can also be ran standalone of course, against a production server for example.
|
Going to merge this, so I can update the server PR with the @latest reference |
derekcollison
added a commit
to nats-io/nats-server
that referenced
this pull request
Feb 6, 2024
This PR adds a number of MQTT-specific benchmarks, executed with an external test client (see ConnectEverything/mqtt-test#1). This came out of my work on MQTT message delivery issues in 2.10, and is now packaged as a `go test --bench`, and added to CI. The test client uses the Eclipse `paho` library, and can be executed standalone, pointed at an MQTT server. It measures the timing of its operations, and outputs it as JSON of reportable metrics. The benchmarks here run the client, and report back the metrics it outputs. Memory allocations are measured server-only. Github action `benchmark` is used to compare the benchmark results to the base ref of the PR, or the prior result in a branch. If it fails a 120% threshold, the failure is reported as a comment on the commit(!), and the build fails. To measure the effect of parallel-fetching retained messages (`SUBRET`), added `disable_retained_message_cache` MQTT option to the server. The following benchmarks are run, covering standalone/cluster, QOS, message sizes to 100K, as well as concurrent clients. - `PUB` measures just publishing - `PUBRET` measures publishing with the retained flag on - `PUBSUB` measures message delivery times from publisher to subscriber - `SUBRET` measures delivery of 100* retained messages to new subscriptions Example of a failure (I rolled back the retained message "fanout" commit): <img width="876" alt="image" src="https://github.com/nats-io/nats-server/assets/1187448/c56eb408-4e4a-4562-ab41-9c024b09ba47"> if you click on the comment link, you get: <img width="1025" alt="image" src="https://github.com/nats-io/nats-server/assets/1187448/2b677bb5-8445-4ed0-af5b-bf8519d42f9f"> ###### TODO - `CON` measure CONNECT to CONNACK, with stored sessions - `SUB` measuring SUBSCRIBE to SUBACK
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For use in nats-io/nats-server#4821