DeepSeek @deepseek_ai

🚀 Day 4 of #OpenSourceWeek: Optimized Parallelism Strategies

✅ DualPipe - a bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.
🔗 https://t.co/GBtxSvWLT4

✅ EPLB - an expert-parallel load balancer for V3/R1.
🔗

github.com

GitHub - deepseek-ai/DualPipe: A bidirectional pipeline parallelism algorithm for computation-com...

A bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training. - deepseek-ai/DualPipe