智谱AI技术团队 2024-08-28 10:22 北京
3060能跑～
8月6日，我们发布并开源了CogVideoX-2B模型，受到广大开发者的欢迎。
为了促进社区的自主使用和开放式创新，我们现决定将参数规模更大、性能更强的产品级模型 CogVideoX-5B 开源，同时 CogVideoX-2B 的开源协议调整为更加开放的Apache 2.0协议。
CogVideoX-2B 升级
自8月6日发布以来，CogVideoX-2B 在推理设备门槛上已有显著优化：
支持多种推理精度，包括 FP16、BF16、FP32、INT8 等；
在 INT8 精度下，推理显存需求仅为 7.8GB，即使用 1080 Ti 显卡也可以完成推理；
在 FP16/BF16 精度，Step=50 的设定下，单卡 A100 仅需 90 秒完成推理，单卡 H100 仅需 45 秒；
LoRA 微调显存需求为 47GB，SFT 微调为 62GB，单卡 A100（80GB）即可满足微调需求。
CogVideoX-5B 开源
CogVideoX-5B 模型相较于 CogVideoX-2B 拥有更高的视频生成质量，并在工程优化上保持了一致的高标准：
同样支持多种推理精度；
推理显存需求最低仅为 11.4GB，即单张 3060 显卡即可完成推理；
LoRA 微调显存需求为 63GB，SFT 微调为 75GB，单卡 A100（80GB）同样可以完成微调任务。
为进一步支持开源社区的广泛使用与开发，我们已将 CogVideoX-2B 的协议调整为 Apache 2.0，任何企业与个人均可自由使用。
同时，我们推出了在线体验，并将其源代码开源，涵盖了插帧、超分辨率等先进功能，均采用开源技术方案，实现全流程开源。
代码仓库：https://github.com/THUDM/CogVideo
模型论文：https://arxiv.org/abs/2408.06072
模型下载：
Huggingface：https://huggingface.co/THUDM/CogVideoX-5b
魔搭社区：https://modelscope.cn/models/ZhipuAI/CogVideoX-5b
在线体验：
Huggingface：https://huggingface.co/spaces/THUDM/CogVideoX-5B-Space
魔搭社区：https://modelscope.cn/studios/ZhipuAI/CogVideoX-5b-demo
下面为模型参数概要：
Prompt：A suited astronaut, with the red dust of Mars clinging to their boots, reaches out to shake hands with an alien being, their skin a shimmering blue, under the pink-tinged sky of the fourth planet. In the background, a sleek silver rocket, a beacon of human ingenuity, stands tall, its engines powered down, as the two representatives of different worlds exchange a historic greeting amidst the desolate beauty of the Martian landscape.
Prompt：A small boy, head bowed and determination etched on his face, sprints through the torrential downpour as lightning crackles and thunder rumbles in the distance. The relentless rain pounds the ground, creating a chaotic dance of water droplets that mirror the dramatic sky's anger. In the far background, the silhouette of a cozy home beckons, a faint beacon of safety and warmth amidst the fierce weather. The scene is one of perseverance and the unyielding spirit of a child braving the elements.
Prompt：On a brilliant sunny day, the lakeshore is lined with an array of willow trees, their slender branches swaying gently in the soft breeze. The tranquil surface of the lake reflects the clear blue sky, while several elegant swans glide gracefully through the still water, leaving behind delicate ripples that disturb the mirror-like quality of the lake. The scene is one of serene beauty, with the willows' greenery providing a picturesque frame for peaceful avian visitors.
Prompt：A Chinese mother, draped in a soft, pastel-colored robe, gently rocks back and forth in a cozy rocking chair positioned in the tranquil setting of a nursery. The dimly lit bedroom is adorned with whimsical mobiles dangling from the ceiling, casting shadows that dance on the walls. Her baby, swaddled in a delicate, patterned blanket, rests against her chest, the child's earlier cries now replaced by contented coos as the mother's soothing voice lulls the little one to sleep. The scent of lavender fills the air, adding to the serene atmosphere, while a warm, orange glow from a nearby nightlight illuminates the scene with a gentle hue, capturing a moment of tender love and comfort.
Prompt：In a dimly lit bar, purplish light bathes the face of a mature man, his eyes blinking thoughtfully as he ponders in close-up, the background artfully blurred to focus on his introspective expression, the ambiance of the bar a mere suggestion of shadows and soft lighting.
Prompt：An elderly gentleman, with a serene expression, sits at the water's edge, a steaming cup of tea by his side. He is engrossed in his artwork, brush in hand, as he renders an oil painting on a canvas that's propped up against a small, weathered table. The sea breeze whispers through his silver hair, gently billowing his loose-fitting white shirt, while the salty air adds an intangible element to his masterpiece in progress. The scene is one of tranquility and inspiration, with the artist's canvas capturing the vibrant hues of the setting sun reflecting off the tranquil sea.
Prompt：A garden comes to life as a kaleidoscope of butterflies flutters amidst the blossoms, their delicate wings casting shadows on the petals below. In the background, a grand fountain cascades water with a gentle splendor, its rhythmic sound providing a soothing backdrop. Beneath the cool shade of a mature tree, a solitary wooden chair invites solitude and reflection, its smooth surface worn by the touch of countless visitors seeking a moment of tranquility in nature's embrace.
Prompt：A golden retriever, sporting sleek black sunglasses, with its lengthy fur flowing in the breeze, sprints playfully across a rooftop terrace, recently refreshed by a light rain. The scene unfolds from a distance, the dog's energetic bounds growing larger as it approaches the camera, its tail wagging with unrestrained joy, while droplets of water glisten on the concrete behind it. The overcast sky provides a dramatic backdrop, emphasizing the vibrant golden coat of the canine as it dashes towards the viewer.
阅读原文，直达“代码仓库”！
