Sync upstream/bach

[bachvudinh]

Issue: Sync Torchtune fork from upstream

Description:

This PR involves syncing the training code with the latest updates from the upstream repository to ensure our codebase remains up-to-date.
Tasks:

• Pull the latest changes from the upstream repository.
• Resolve any merge conflicts.
• Test the synchronized code to ensure all functionality remains intact.
• Update dependent files or configurations if necessary.

Expected Outcome:

The training code will be synchronized with the upstream repository, ensuring we are aligned with the latest features, improvements, and bug fixes.

Some Major new Feature

• Qwen2.5 model family has landed (Qwen2.5 pytorch/torchtune#1863)

  • added builders for Qwen2.5, the cutting-edge models from the Qwen family of models! In their own words "Compared to Qwen2, Qwen2.5 has acquired significantly more knowledge (MMLU: 85+) and has greatly improved capabilities in coding (HumanEval 85+) and mathematics (MATH 80+)."

• Change the Lr scheduler module path to torch.training.lr_scheduler

• added model builders and configs for the 90B version of Llama3.2V

• and fix some bug here for more details: https://github.com/pytorch/torchtune/releases.

Test plan

Running some training with Ichigo Llama3.1-s and MiniIchigo llama3.2-s. ALso do some testing relate to vram spike when do distributed training.