add use_all_gather for option

[SangbumChoi]

What does this PR do?

Fixes #3147
@muellerzr

I think testing might need but not sure where to add this corresponding test

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[SangbumChoi]

Also have additional question for LLM inference evaluation process. Does LLM evaluation process requires to have all_gather function to make all the prediction in all processes?

[muellerzr]

Thanks! Tests should be in src/accelerate/test_utils/scripts/test_ops.py

The all_gather is so that we can get the results across all processes. It's there as an example and for simplicity for the user. (But it is not generally required, no)

[SangbumChoi]

@muellerzr Hi I haved added test. However, to be more strict test I think we can add additional checking when the process is not in the state.is_main_process and check the return is empty list. Let me know how do you think about this.

[SangbumChoi]

@muellerzr Gentle Ping!

[muellerzr]

Since using all_gather uses more memory by default, let's set a default to False please (even though we said the opposite initially I think)

[muellerzr]

re; test:I think it's fine how it is right now

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.