Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More fine-grained assignment of resources to tasks in MapReduceJob #309

Closed
AnesBenmerzoug opened this issue Mar 9, 2023 · 2 comments
Closed
Assignees
Labels
enhancement New feature or request utils
Milestone

Comments

@AnesBenmerzoug
Copy link
Collaborator

AnesBenmerzoug commented Mar 9, 2023

In order to have proper interaction with within-node parallelization of utility computations and avoid cpu contention, we need to be able to configure remote tasks to run e.g. one per node, or to limit the number of cores (and ram and gpus) that they see.

@AnesBenmerzoug AnesBenmerzoug added enhancement New feature or request utils labels Mar 9, 2023
@AnesBenmerzoug AnesBenmerzoug added this to the v0.7.0 milestone Mar 9, 2023
@AnesBenmerzoug AnesBenmerzoug self-assigned this Sep 7, 2023
@mdbenito
Copy link
Collaborator

We might want to drop MapReduce altogether (think of issues with convergence criteria)

@mdbenito mdbenito modified the milestones: v0.8.0, v0.9.0 Oct 26, 2023
@schroedk
Copy link
Collaborator

schroedk commented Sep 9, 2024

Usage of MapReduce is deprecated, fine grain config is achieved in #558 via joblib.

@schroedk schroedk closed this as completed Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request utils
Projects
None yet
Development

No branches or pull requests

3 participants