Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Batching Collector functionality for LdioSender component #662

Open
Yalz opened this issue Jul 5, 2024 · 1 comment
Open

Make Batching Collector functionality for LdioSender component #662

Yalz opened this issue Jul 5, 2024 · 1 comment
Labels
component: ldio Issues related to Linked Data Interactions move to JIRA

Comments

@Yalz
Copy link
Collaborator

Yalz commented Jul 5, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.
To reduce network traffic for LDIO outputs, we should move batch collecting from the Repository Sink upwards to all LDIO Output components.

Describe the solution you'd like
At the LdioSender, collect members until they either reach a batch threshold or a timeout. After that, send them to our Outputs as combined model (to discuss on this decision)
@Yalz Yalz added the needs triage Issue needs to be evaluated by team label Jul 5, 2024
@Yalz Yalz changed the title Make Batching Collector functionality for all LdioOut components Make Batching Collector functionality for all LdioSender components Jul 5, 2024
@Yalz Yalz added this to VSDS Backlog Jul 5, 2024
@Yalz Yalz changed the title Make Batching Collector functionality for all LdioSender components Make Batching Collector functionality for all LdioSender Jul 5, 2024
@github-project-automation github-project-automation bot moved this to 📋 Backlog in VSDS Backlog Jul 5, 2024
@Yalz Yalz changed the title Make Batching Collector functionality for all LdioSender Make Batching Collector functionality for LdioSender component Jul 5, 2024
@Yalz Yalz added component: ldio Issues related to Linked Data Interactions and removed needs triage Issue needs to be evaluated by team labels Jul 5, 2024
@Yalz
Copy link
Collaborator Author

Yalz commented Jul 18, 2024

To be detailed in https://vlaamseoverheid.atlassian.net/browse/VSDSPUB-1415

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component: ldio Issues related to Linked Data Interactions move to JIRA
Projects
VSDS Backlog
Status: 📋 Backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Yalz