IP属地：青海
-
LLaMA：开源的高效的基础语言模型LLaMA: Open and Efficient Foundation Language Models Feb 2023 Hugo Touvr...
-
缩放律与模型架构：归纳偏置如何影响缩放？
Scaling Laws vs Model Architectures: How does Inductive Bias Influence S...
-
UL2：统一语言学习范式
UL2: Unifying Language Learning Paradigms https://arxiv.org/abs/2205.051...
-
用0.1%的额外计算超越缩放定律
Transcending Scaling Laws with 0.1% Extra Compute https://arxiv.org/abs/...
-
大语言模型的涌现能力
Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.0...
-
预训练指南：测量数据年龄、领域覆盖率、质量和毒性的影响
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age...
-
自回归生成建模的缩放律
Scaling Laws for Autoregressive Generative Modeling Oct 2020 https://arx...
-
神经语言模型的缩放律
Scaling Laws for Neural Language Models Jan 2020 https://arxiv.org/abs/2...
-
DoReMi：优化数据混合加速语言模型预训练
预训练数据域（如维基百科、书籍、网络文本）的混合比例极大地影响了语言模型（LM）的性能。在本文中，我们提出了具有Minimax优化的域重新加权（...