Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why the ProcessAgent use Process while the ThreadTrainer use Thread? #38

Open
tangbohu opened this issue Jul 5, 2018 · 2 comments
Open

Comments

@tangbohu
Copy link

tangbohu commented Jul 5, 2018

Hello!

Why the ProcessAgent use Process while the ThreadTrainer use Thread? I wonder whether the ProcessAgent.py could use Thread instead of Process

@boluoweifenda
Copy link

I have the same question. Is it because the efficiency of data transform in Queue between processes?

@boluoweifenda
Copy link

I might figure out the reason after reimplementing this project.
Since the trainers and predictors have the same tensorflow model and parameters, and it's hard/inefficient to share them by Queue in multiprocess class.
Although in Python the multithread class cannot use multi-core due to GIL, the trainer and predictors are usually using the same GPU, so the bottleneck happens in GPU computation rather than in the multithread trainers and predictors in the main process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants