Foundations of LLM Mastery: Fine-tuning on multi GPUs

This course takes your model training to the next level by scaling across multiple GPUs. We cover distributed training strategies – including DDP, ZeRO, and FSDP – to help you handle larger datasets and more complex models with speed and efficiency.
Key topics include:
Distributed Data Parallel (DDP): Efficiently distribute data and model updates across multiple GPUs.
ZeRO with DeepSpeed: Scale large models while managing memory effectively.
Fully Sharded Data Parallel (FSDP): Yet another technique to reduce memory usage by sharding.
Hugging Face Accelerate: Simplify multi-GPU training with intuitive tools and configurations.