Make scripts/download_datasets.py
download BEIR datasets
#13
Labels
enhancement
New feature or request
scripts/download_datasets.py
download BEIR datasets
#13
In
Dockerfile
, we call the scriptscripts/download_datasets.py
that downloads all datasets to/var/tmp/pv211
, so that the datasets are shared by all students that use JupyterHub, saving time and disk space. For example, here we download the ARQMath datasets: 1->2. Here, the students load them: 3->4->5.Since #3, we've supported BEIR datasets. However, the BEIR datasets are not downloaded in
Dockerfile
and they are saved and loaded from the./datasets
directory, which slows down the students and duplicates disk space occupied.Tasks
/var/tmp/pv211
inscripts/download_datasets.py
./var/tmp/pv211
inpv211_utils.beir.loader
.The text was updated successfully, but these errors were encountered: