Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support launching Task Server as Thread, not Process #49

Open
WardLT opened this issue Jan 27, 2022 · 1 comment
Open

Support launching Task Server as Thread, not Process #49

WardLT opened this issue Jan 27, 2022 · 1 comment

Comments

@WardLT
Copy link
Collaborator

WardLT commented Jan 27, 2022

Avoid copying between the Task Server and Thinker may outweigh the need to avoid the GIL in some cases. I have the FuncXTaskServer clearly in mind here, where the only think it does is communicate to another server.

As pointed out by @gpauloski , shared memory between the Task Server/Thinker will be especially useful as we can start resolving object proxies as soon as they are received by the task server.

We will need to implement Thread Queue support (#36) before addressing this issue.

@WardLT
Copy link
Collaborator Author

WardLT commented Nov 28, 2023

An idea on implementation that will also open up #119 : Do not subclass Process but, instead, provide a start_server method which takes a queue as an argument. This will fix the issues with have with multiprocessing queues and pickling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant