-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Process Commands #198
Process Commands #198
Conversation
We don't expect users to be changing public Process attributes like Process.parameters. This commit introduces public getters to communicate that these are read-only attributes.
Instead of holding ParallelProcess objects in a separate Engine.parallel dictionary, put them in place of their associated Process objects in Engine.processes, Engine.process_paths, and Engine.state. This also involves removing InvokeProcess and the `invoke` keyword from the Engine constructor. While these are public, the `invoke` keyword was not documented, and use of InvokeProcess is not discussed in our documentation. Therefore, neither are part of our supported API, so removing them does not constitute a breaking change.
Requires PR vivarium-collective/vivarium-core#198, which introduces process commands. The process commands interface lets us retrieve internal state from parallelized EngineProcess instances.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking really good! I have no improvements. Engine
is actually simpler now in many ways. Process
is a bit more complicated, but I think it all makes sense and is required.
@prismofeverything -- Can you give this a look, since it is a major design decision?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey Chris, this is an excellent feature! I think we should keep it, and treat our processes as "services" essentially in this way, which sets the groundwork for a fully distributed implementation. Some minor comments but really it's good to go.
With the move to Process Commands, we stopped parallelizing processes in `Engine._invoke_process` and instead performed parallelization in the constructor. This had the unintended effect of not parallelizing processes with the `parallel` attribute if they were produced by division. This commit ensures that these processes are parallelized as well.
Introduce process commands to support more interactions with parallel processes. Now all
Process
methods of a parallelized process can be queried from the parent OS process. Users can also add support for custom methods of their processes.This change also simplifies the way
Engine
handles parallel processes warns users when serializers are not being found efficiently.TODO:
Engine
to putParallelProcess
instances in the store hierarchyinvoke_process
function that doesn't do anythingBy creating this pull request, I agree to the Contributor License
Agreement, which is available in
CLA.md
at the top level of thisrepository.