DeepSeek前实习生魔改MoE，用迭代机制把内存需求砍了42%，团队：“免费午餐”优化方法
"内存效率" 相关文章
Meet Tensor Product Attention (TPA): Revolutionizing Memory Efficiency in Language Models
Researchers from Caltech, Meta FAIR, and NVIDIA AI Introduce Tensor-GaLore: A Novel Method for Efficient Training of Neural Networks with Higher-Order Tensor Weights
Meet CoMERA: An Advanced Tensor Compression Framework Redefining AI Model Training with Speed and Precision
Optimizing AI Models with Quanto on H100 GPUs
基于 Quanto 和 Diffusers 的内存高效 transformer 扩散模型
Q-GaLore Released: A Memory-Efficient Training Approach for Pre-Training and Fine-Tuning Machine Learning Models
Adam-mini: A Memory-Efficient Optimizer Revolutionizing Large Language Model Training with Reduced Memory Usage and Enhanced Performance
Training on a Dime: MEFT Achieves Performance Parity with Reduced Memory Footprint in LLM Fine-Tuning
