Skip to content

Commit

Permalink
Add tqdm progress bar to dials.import (cctbx#768)
Browse files Browse the repository at this point in the history
Give the user some hint as to the progress i.e. the computer is doing
something when you import thousands of files.

Only show progress bar if > 1 file and isatty, includes a cast of the `filenames` iterator to a list
to allow evaluation of the length for progress assessment.

Resolves dials/dials#2783
  • Loading branch information
graeme-winter authored Nov 25, 2024
1 parent ea6fd5f commit 56b467b
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 1 deletion.
2 changes: 2 additions & 0 deletions newsfragments/768.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
``dials.import``: add a progress bar for the import process, particularly
useful when importing thousands of files
12 changes: 11 additions & 1 deletion src/dxtbx/model/experiment_list.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from typing import Any, Callable, Generator, Iterable

import natsort
from tqdm import tqdm

import dxtbx
from dxtbx.format.Format import Format
Expand Down Expand Up @@ -664,14 +665,23 @@ def from_filenames(
"""Create a list of data blocks from a list of directory or file names."""
experiments = ExperimentList()

# Cast filenames to a list from whatever iterator they are
filenames = list(filenames)

# Process each file given by this path list
to_process = _openingpathiterator(filenames)
find_format = FormatChecker()

format_groups = collections.defaultdict(list)
if format_kwargs is None:
format_kwargs = {}
for filename in to_process:

if os.isatty and len(filenames) > 1:
filename_iter = tqdm(to_process, total=len(filenames))
else:
filename_iter = to_process

for filename in filename_iter:
# We now have a file, pre-opened by Format.open_file (therefore
# cached). Determine its type, and prepare to put into a group
format_class = find_format.find_format(filename)
Expand Down

0 comments on commit 56b467b

Please sign in to comment.