Last image is not building

[lukaspetersson]

Describe the bug

I followed the setup and ran:

python -m swebench.harness.run_evaluation \
    --predictions_path gold \
    --max_workers 4 \
    --run_id validate-gold

It starts building 40 docker images, but get stuck on the last one. I waited several hours and tried it on many different machines.

I checked logs/build_images/base/sweb.base.x86_64__latest/build_image.log but could not find anything that seemed wrong.

Steps/Code to Reproduce

git clone [email protected]:princeton-nlp/SWE-bench.git
cd SWE-bench
pip install -e .

python -m swebench.harness.run_evaluation \
    --predictions_path gold \
    --max_workers 4 \
    --run_id validate-gold

Expected Results

All images build

Actual Results

Last image just loads for ever

System Information

Tried on a fedora linux (32gb RAM), a m1 MacBook pro, and a debian VM (64gb ram)

[zyone1991]

Faced the same issue before. One thing you might want to try is change max_workers to 1 in your provided command and run it multiple times. It eventually took 2-3 times for me to build the environment image. I suspect this is related to max_workers but I cannot figure out why this happens.