Timeout Error

[Dormiveglia-elf]

ERROR:ragas.executor:Exception raised in Job[2]: TimeoutError()
ERROR:ragas.executor:Exception raised in Job[7]: TimeoutError()
ERROR:ragas.executor:Exception raised in Job[10]: TimeoutError()
ERROR:ragas.executor:Exception raised in Job[15]: TimeoutError()
....

I use GPT-4o and encountered many timeout error, any ideas?

[Dormiveglia-elf]

I have reduced the number of timeout error by adjusting the max_workers to 60 and time_out parameter to 120 mentioned in https://docs.ragas.io/en/stable/howtos/customizations/_run_config/?h=time

But still some error existing, I think 120 is a very long time. Any ideas?

[YuffieHuang]

max_workers

Further reducing max_workers solved the same issue for me. I set it to be 8 for my locally hosted models.

[jjmachan]

hey anothe approach is batch_size argument for evalaute() https://docs.ragas.io/en/stable/references/evaluate/#ragas.evaluation.evaluate

something else you could try? easier to reason about IMO

[Sergio-Zavota]

reducing max_workers to 10 and increasing the timeout to 120 seconds also worked for me. Even with a 120-second timeout, I could still get a parallelism of 8-9 out of 10, which might indicate that just a few workers were taking a long time to process.