Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New pub/sub endpoint for labs who need just build/checkout events #557

Open
nuclearcat opened this issue Oct 22, 2024 · 0 comments
Open

Comments

@nuclearcat
Copy link
Member

After recent discussion we decided that we need a bit more easier mechanism to subscribe for new events, or even just queue of newly created nodes. I propose following:

  • We use https://www.mongodb.com/docs/manual/core/index-ttl/ to remove more than N day old nodes in collection "events". This way we keep queue small. I propose to set N to 1 day

  • On startup we read last event counter, example name "last_event_id" , likely stored somewhere in mongodb, collection "preferences" or something like that, or maybe we keep preferences in redis. I believe redis with persistent storage should be better (faster).

  • When we receive event, we create new document, and assign to it ID = last event ID + 1, and update last_event_id in preferences.

  • we have endpoint /getevents?from=ID1&to=ID2, which returns all events (nodes) with IDs from ID1 to ID2. "to" is optional,
    if not provided, return all events from ID1 to the last event. Also we can add option limit and simple filter by event type (checkout, kbuild, etc).

This will make for labs trivial to receive new events. They just need to store last event ID, and call /getevents?from=last_event_id+1&filterkind=kbuild
If there are no events, they will just get empty list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant