Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement engine="thread" in ChunkRecordingExecutor #3526

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Implement engine="thread" in ChunkRecordingExecutor #3526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e0ef39b
proof of concept of chunkexecutor with thread
samuelgarcia Nov 8, 2024
a28c33d
for progress_bar the for res in results need to be inside the with
samuelgarcia Nov 8, 2024
67b055b
wip
samuelgarcia Nov 8, 2024
cecb211
wip
samuelgarcia Nov 12, 2024
4365321
Move worker_index to job_tools.py
samuelgarcia Nov 20, 2024
e929820
change max_threads_per_process to max_threads_per_worker
samuelgarcia Nov 20, 2024
d4a6e95
implement get_best_job_kwargs()
samuelgarcia Nov 20, 2024
423801b
oups
samuelgarcia Nov 20, 2024
1736b65
oups
samuelgarcia Nov 20, 2024
f0ec139
Feedback from Zach and Alessio better test for waveforms_tools
samuelgarcia Nov 22, 2024
cc8b4c4
fix tests
samuelgarcia Nov 22, 2024
5cf6ace
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia Nov 22, 2024
c16ca72
oups
samuelgarcia Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/get_started/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ available parameters are dictionaries and can be accessed with:
'detect_threshold': 5,
'freq_max': 5000.0,
'freq_min': 400.0,
'max_threads_per_process': 1,
'max_threads_per_worker': 1,
'mp_context': None,
'n_jobs': 20,
'nested_params': None,
Expand Down
2 changes: 1 addition & 1 deletion src/spikeinterface/core/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@
write_python,
normal_pdf,
)
from .job_tools import ensure_n_jobs, ensure_chunk_size, ChunkRecordingExecutor, split_job_kwargs, fix_job_kwargs
from .job_tools import get_best_job_kwargs, ensure_n_jobs, ensure_chunk_size, ChunkRecordingExecutor, split_job_kwargs, fix_job_kwargs
from .recording_tools import (
write_binary_recording,
write_to_h5_dataset_format,
Expand Down
2 changes: 1 addition & 1 deletion src/spikeinterface/core/globals.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def is_set_global_dataset_folder() -> bool:


########################################
_default_job_kwargs = dict(n_jobs=1, chunk_duration="1s", progress_bar=True, mp_context=None, max_threads_per_process=1)
_default_job_kwargs = dict(pool_engine="thread", n_jobs=1, chunk_duration="1s", progress_bar=True, mp_context=None, max_threads_per_worker=1)

global global_job_kwargs
global_job_kwargs = _default_job_kwargs.copy()
Expand Down
Loading