Feature/col 3 rewind #62
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zknill
reviewed
Oct 19, 2023
zknill
reviewed
Oct 19, 2023
mschristensen
force-pushed
the
feature/COL-3-rewind
branch
from
772268f to
27d6aa7
Compare
Adds a SequenceResumer middleware that discards messages up until the attach point given by the sequence ID is reached. Can be used in conjunction with a sliding window re-orderer via the OrderedSequenceResumer middleware.
The SyncFunc now returns the sequenceID and stateTimestamp along with the model state snapshot. This is used by the model to rewind to the correct point in the stream.
In order to work around rewind's 100 message limit, we can use a history query with untilAttach to paginate back through messages until the target sequenceID boundary is reached. This has the additional benefit that enabling persisted history on the channel just works, allowing you to extend your state cache TTL.
mschristensen
force-pushed
the
feature/COL-3-rewind
branch
from
322e652 to
f65b0e7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(Closed #55 as the branch name was misleading - we have not ended up using an interval-based rewind).
In fact, we aren't actually using rewind; but it's described as such in the ticket so keeping that naming here.
Instead we can use history with
untilAttachto paginate back through messages to find the sequenceID boundary from which to replay messages. This can be done while preserving continuity with incoming live messages.Other point of note: since with adbc the message ID is actually the SequenceID (a serial number) but the message ID type is a string, lexicographical ordering doesn't work for us ('2' > '10' in this case). So, we sort numerically if we can and fallback to lexicographical order when we can't.