-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add queueing for async actions #464
Comments
I'd like to work on this. But I'm a little confused about how the action |
Hey @Hamsajj, @likecodingloveproblems asked me earlier on LinkedIn to work on this same exact ticket, but we decided that it'd be best for him to start with another ticket to get to know the codebase, which he did in #503. I'd be very happy to give this to either or both of you. On the other hand, I know that you worked on more tickets, especially those related to the Act system, and you have a good grasp of what it entails. Alternatively, @likecodingloveproblems might want to choose another ticket of the same epic if he wants. I'd really like to know what both of you think about what I mentioned, so we can proceed. @Hamsajj I'll update the ticket description to answer your technical questions about this ticket. |
I can happily work on another ticket if @likecodingloveproblems wants to work on this one. As you mentioned, there are a lot of tickets on the same act system epic. (btw, kudos to you @mostafa for setting up these issues and tickets, making this repo welcoming and relatively easy to contribute) Also, thanks @mostafa for adding the detailed explanation of how the |
Hey @Hamsajj, I know @likecodingloveproblems is busy these days, so I suggest picking this up if you like. |
Hey @Hamsajj, Is this still in progress? Do you need more context? |
Hi @mostafa |
No rush! I was mostly interested in seeing progress, not pushing. 🙏 |
Currently the asynchronous actions run as a goroutine. This should be changed to using a worker that queues the outputs for async actions and run them separately. This can be done as a separate command, like
gatewayd worker
, that run a set of workers that consume from a message queue thatgatewayd run
produces messages to.The idea is that if an action is async, it can be defined in a worker and registered in a registry with a pointer to the worker. The worker contains a
Run
function, which executes the action. Data needed for theRun
function is dispatched (published) to the worker via a queue, enabling theRun
function's execution. Results or errors are returned through an alternate queue set back to the Act system. For this, we don't need to change the implementation of the action, rather we can have a generic queue message publisher that implements theActionFunc
, which can be reused in async actions for publishing the exported Action fields to the queue. The message will be picked up by the worker to execute theRun
function.I have implemented a minimal example in the act-poc repository for queueing using this channel, which is used in action like this and then the results are logged into the terminal. It uses the awesome golang-queue project, which I recommend to be used as high-level wrapper for our use case.
This is what I have on my mind:
Resources
The text was updated successfully, but these errors were encountered: