⏶121
Seaweed-7B：视频生成基础模型的经济高效训练
发表
由 
Lu Jiang 提交
Lu Jiang 提交作者: Team Seawead, Ceyuan Yang, 
Zhijie Lin, Yang Zhao, Shanchuan Lin, 
Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, 
Feng Cheng, Feilong Zuo Xuejiao Zeng, 
Ziyan Yang, Fangyuan Kong, Zhiwu Qing, Fei Xiao, Meng Wei, Tuyen Hoang, Siyu Zhang, Peihao Zhu, Qi Zhao, Jiangqiao Yan, Liangke Gui, Sheng Bi, Jiashi Li, Yuxi Ren, Rui Wang, Huixia Li, Xuefeng Xiao, Shu Liu, 
Feng Ling, Heng Zhang, Houmin Wei, Huafeng Kuang, Jerry Duncan, Junda Zhang, Junru Zheng, Li Sun, Manlin Zhang, Renfei Sun, 
Xiaobin Zhuang, Xiaojie Li, Xin Xia, Xuyan Chi, Yanghua Peng, Yuping Wang, Yuxuan Wang, Zhongkai Zhao, Zhuo Chen, Zuquan Song, 
Zhenheng Yang, Jiashi Feng, Jianchao Yang, Lu Jiang
Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, Feng Cheng, Feilong Zuo Xuejiao Zeng, Ziyan Yang, Fangyuan Kong, Zhiwu Qing, Fei Xiao, Meng Wei, Tuyen Hoang, Siyu Zhang, Peihao Zhu, Qi Zhao, Jiangqiao Yan, Liangke Gui, Sheng Bi, Jiashi Li, Yuxi Ren, Rui Wang, Huixia Li, Xuefeng Xiao, Shu Liu, Feng Ling, Heng Zhang, Houmin Wei, Huafeng Kuang, Jerry Duncan, Junda Zhang, Junru Zheng, Li Sun, Manlin Zhang, Renfei Sun, Xiaobin Zhuang, Xiaojie Li, Xin Xia, Xuyan Chi, Yanghua Peng, Yuping Wang, Yuxuan Wang, Zhongkai Zhao, Zhuo Chen, Zuquan Song, Zhenheng Yang, Jiashi Feng, Jianchao Yang, Lu Jiang摘要
本技术报告介绍了一种经济高效的策略，用于训练视频生成基础模型。我们提出了一个中等规模的研究模型，约有 70 亿参数 (7B)，名为 Seaweed-7B，使用 665,000 H100 GPU 小时从头开始训练。尽管 Seaweed-7B 使用适中的计算资源进行训练，但与当代规模更大的视频生成模型相比，它表现出极具竞争力的性能。在资源受限的环境中，设计选择尤为关键。本技术报告重点介绍了提升中等规模扩散模型性能的关键设计决策。根据经验，我们得出两个观察结果：（1）Seaweed-7B 的性能与使用更多 GPU 资源训练的更大模型相当，甚至超越了它们；（2）我们的模型展现出强大的泛化能力，可以通过轻量级微调或持续训练有效地适应各种下游应用。项目主页请见 https://seaweed.video/
Morice Michel
eepos
Michael Barry
SivaNithish
PseudoTerminal X
qiang
Seaweed-7B：一个经济高效的视频生成基础模型