Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #8829] Support for Persisting LMQ Consumer Offsets in Config V1 Using RocksDB #8939

Merged
merged 1 commit into from
Nov 18, 2024

Conversation

RongtongJin
Copy link
Contributor

@RongtongJin RongtongJin commented Nov 16, 2024

Which Issue(s) This PR Fixes

Fixes #8829

In the current v1, lmq consumer offsets are only stored in memory, which means they will be lost if the broker restarts or crashes. To address this, #8856 @lizhanhui developed v2 to enable storing lmq consumer offsets in RocksDB. However, upgrading from v1 to v2 with backward compatibility is not feasible. Therefore, it's crucial to introduce support for persisting lmq consumer offsets in v1.

Brief Description

To streamline the implementation, we will not use a separate class for lmq consumer offset storage. Instead, we will leverage the existing RocksDBConsumerOffsetManager. Under the v1 metadata configuration, this manager will handle both lmq and non-lmq consumer offsets collectively.

How Did You Test This Change?

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 47.55%. Comparing base (4574647) to head (e533a1a).

Files with missing lines Patch % Lines
...a/org/apache/rocketmq/broker/BrokerController.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8939      +/-   ##
=============================================
+ Coverage      47.53%   47.55%   +0.02%     
+ Complexity     11731    11724       -7     
=============================================
  Files           1304     1303       -1     
  Lines          91058    91023      -35     
  Branches       11675    11671       -4     
=============================================
+ Hits           43288    43290       +2     
+ Misses         42426    42390      -36     
+ Partials        5344     5343       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DongyuanPan DongyuanPan merged commit 163451e into apache:develop Nov 18, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] LMQ Optimization: Revamp on top of RocksDB
5 participants