v0.2.0 SageMaker launcher

v0.2.0 SageMaker launcher

SageMaker launcher

It's now possible to launch your training script on AWS instances using SageMaker via accelerate launch.

• Launch script on SageMaker #26 (@philschmid )
• Add defaults for compute_environmnent #23 (@sgugger )
• Add Configuration setup for SageMaker #17 (@philschmid )

Kwargs handlers

To customize how the different objects used for mixed precision or distributed training are instantiated, a new API called KwargsHandler is added. This allows the user to pass along the kwargs that will be passed to those objects if used (and it is ignored if those are not used in the current setup, so the script can still run on any kind of setup).

• Add KwargsHandlers #15 (@sgugger )

Pad across processes

Trying to gather tensors that are not of the same size across processes resulted in a process hang, a new method Accelerator.pad_across_processes has been added to help with that.

• Add utility to pad tensor across processes to max length #19 (@sgugger )

Various bug fixes

• added thumbnail #25 (@philschmid )
• Cleaner diffs in README and index #22 (@sgugger )
• Use proper size #21 (@sgugger )
• Alternate diff #20 (@sgugger )
• Add YAML config support #16 (@sgugger )
• Don't error on non-Tensors objects in move to device #13 (@sgugger )
• Add CV example #10 (@sgugger )
• Readme clean-up #9 (@thomwolf )
• More flexible RNG synchronization #8 (@sgugger )
• Fix typos and tighten grammar in README #7 (@lewtun )
• Update README.md #6 (@voidful )
• Fix TPU training in example #4 (@thomwolf )
• Fix example name in README #3 (@LysandreJik )