2407.21783
Jul 31, 2024
cs.AI
Llama 3 모델군(The Llama 3 Herd of Models)
The Llama 3 Herd of Models
Abhimanyu Dubey
Abhimanyu Dubey
Facebook AI Research
Citations:
15,690
h-index:
24
Amy Yang
Amy Yang
Citations:
13,403
h-index:
8
Angela Fan
Angela Fan
Citations:
13,689
h-index:
6
Aobo Yang
Aobo Yang
Citations:
13,368
h-index:
8
A. Korenev
A. Korenev
Citations:
29,265
h-index:
8
Arun Rao
Arun Rao
Citations:
13,257
h-index:
6
B. Biron
B. Biron
Citations:
13,758
h-index:
11
Binh Tang
Binh Tang
Citations:
13,447
h-index:
8
Chloe Bi
Chloe Bi
Citations:
13,352
h-index:
8
C. Touret
C. Touret
Citations:
13,509
h-index:
8
C. Wong
C. Wong
Citations:
13,499
h-index:
7
D. Song
D. Song
Citations:
13,543
h-index:
9
D. Pintz
D. Pintz
Citations:
13,261
h-index:
6
Diego Garcia-Olano
Diego Garcia-Olano
University of Texas at Austin
Citations:
13,507
h-index:
6
E. Smith
E. Smith
Citations:
13,397
h-index:
8
G. Nail
G. Nail
Citations:
13,316
h-index:
7
G. Mialon
G. Mialon
Citations:
15,104
h-index:
12
Hu Xu
Hu Xu
Citations:
13,278
h-index:
7
Jade Copet
Jade Copet
Citations:
20,218
h-index:
22
Jaewon Lee
Jaewon Lee
Citations:
13,236
h-index:
6
Jason Park
Jason Park
Citations:
13,254
h-index:
6
Jeet Shah
Jeet Shah
Citations:
13,221
h-index:
5
J. Billock
J. Billock
Citations:
13,231
h-index:
5
Jenny Hong
Jenny Hong
Citations:
13,226
h-index:
5
Jenya Lee
Jenya Lee
Citations:
29,391
h-index:
8
J. Fu
J. Fu
Citations:
29,391
h-index:
8
Jiawen Liu
Jiawen Liu
Citations:
13,239
h-index:
6
Jie Wang
Jie Wang
Citations:
13,249
h-index:
6
Jiecao Yu
Jiecao Yu
Citations:
13,326
h-index:
7
Joe Spisak
Joe Spisak
Citations:
14,766
h-index:
7
K. Upasani
K. Upasani
Citations:
14,324
h-index:
10
Keqian Li
Keqian Li
Citations:
26,379
h-index:
7
L. Maaten
L. Maaten
Citations:
87,135
h-index:
51
Liang Tan
Liang Tan
Citations:
13,593
h-index:
8
Lubo Malo
Lubo Malo
Citations:
13,216
h-index:
5
M. Muzzi
M. Muzzi
Citations:
13,217
h-index:
5
Mannat Singh
Mannat Singh
Facebook AI Research (FAIR)
Citations:
18,361
h-index:
14
Mike Lewis
Mike Lewis
Citations:
13,593
h-index:
8
Min Si
Min Si
Citations:
13,149
h-index:
4
Pengwei Li
Pengwei Li
Citations:
13,570
h-index:
7
Peter Weng
Peter Weng
Citations:
13,230
h-index:
5
P. Dubal
P. Dubal
Citations:
13,260
h-index:
6
Puxin Xu
Puxin Xu
Citations:
30,656
h-index:
9
Qing He
Qing He
Citations:
13,196
h-index:
5
Rohit Girdhar
Rohit Girdhar
Facebook AI Research
Citations:
27,135
h-index:
30
Ruan Silva
Ruan Silva
Citations:
29,291
h-index:
8
Rui Hou
Rui Hou
Citations:
13,847
h-index:
8
Rui Wang
Rui Wang
Citations:
13,512
h-index:
6
Sean Bell
Sean Bell
Citations:
13,598
h-index:
5
S. Kim
S. Kim
Citations:
13,129
h-index:
2
Sergey Edunov
Sergey Edunov
Facebook AI Research
Citations:
45,670
h-index:
19
Sheng Shen
Sheng Shen
University of California, Berkeley
Citations:
24,109
h-index:
33
Shun Zhang
Shun Zhang
Citations:
13,247
h-index:
7
S. Collot
S. Collot
Citations:
13,232
h-index:
5
Suchin Gururangan
Suchin Gururangan
Allen Institute for AI
Citations:
22,464
h-index:
24
T. Fowler
T. Fowler
Citations:
13,760
h-index:
7
Tong Xiao
Tong Xiao
Citations:
13,247
h-index:
6
Ujjwal Karn
Ujjwal Karn
Facebook
Citations:
13,296
h-index:
5
Weiwei Chu
Weiwei Chu
Citations:
13,248
h-index:
6
Wenyin Fu
Wenyin Fu
Citations:
29,392
h-index:
8
Xuchao Jia
Xuchao Jia
Citations:
13,162
h-index:
3
Yiqian Wen
Yiqian Wen
Citations:
13,563
h-index:
4
Yiwen Song
Yiwen Song
Citations:
15,193
h-index:
4
Yuchen Zhang
Yuchen Zhang
Meta AI
Citations:
29,350
h-index:
8
Yue Li
Yue Li
Citations:
13,355
h-index:
7
Yuning Mao
Yuning Mao
Citations:
14,262
h-index:
9
Abha Jain
Abha Jain
Citations:
13,232
h-index:
5
A. Menon
A. Menon
Citations:
13,279
h-index:
6
Anam Yunus
Anam Yunus
Citations:
13,199
h-index:
4
Andrei Lupu
Andrei Lupu
University of Oxford, Meta AI
Citations:
13,751
h-index:
10
A. Caples
A. Caples
Citations:
13,267
h-index:
5
Andrew Gu
Andrew Gu
Citations:
13,192
h-index:
3
Andrew Ho
Andrew Ho
Citations:
13,214
h-index:
5
Beau James
Beau James
Citations:
13,216
h-index:
5
Ben Maurer
Ben Maurer
Citations:
13,218
h-index:
5
Beth Loyd
Beth Loyd
Citations:
13,218
h-index:
5
Bing Liu
Bing Liu
Citations:
15,318
h-index:
9
Bo Wu
Bo Wu
Citations:
13,193
h-index:
4
B. Ni
B. Ni
Citations:
13,212
h-index:
5
Bram Wasti
Bram Wasti
Citations:
14,006
h-index:
7
Chao Zhou
Chao Zhou
Citations:
13,204
h-index:
4
Chester Hu
Chester Hu
Citations:
13,197
h-index:
4
Chris Cai
Chris Cai
Citations:
13,247
h-index:
7
D. Beaty
D. Beaty
Citations:
13,197
h-index:
4
David Xu
David Xu
Citations:
13,190
h-index:
4
Didem Foss
Didem Foss
Citations:
13,166
h-index:
2
Duc Le
Duc Le
Citations:
15,064
h-index:
23
Emily Hahn
Emily Hahn
Citations:
13,193
h-index:
4
Emily Wood
Emily Wood
Citations:
13,193
h-index:
4
E. Arcaute
E. Arcaute
Citations:
13,967
h-index:
6
Fei Sun
Fei Sun
Citations:
13,303
h-index:
6
F. Kreuk
F. Kreuk
Citations:
15,641
h-index:
19
Feng Tian
Feng Tian
Citations:
13,210
h-index:
5
Han Zou
Han Zou
Citations:
13,208
h-index:
5
Han Zha
Han Zha
Citations:
13,138
h-index:
2
Helen Suk
Helen Suk
Citations:
14,085
h-index:
2
Itai Gat
Itai Gat
Meta
Citations:
18,394
h-index:
21
Jeff Tang
Jeff Tang
Citations:
13,134
h-index:
2
J. Chan
J. Chan
Citations:
13,152
h-index:
3
Jenny Zhen
Jenny Zhen
Citations:
13,151
h-index:
3
J. Teboul
J. Teboul
Citations:
13,152
h-index:
3
J. Zhong
J. Zhong
Citations:
13,151
h-index:
3
Jian Jin
Jian Jin
Citations:
13,309
h-index:
6
J. McPhie
J. McPhie
Citations:
13,192
h-index:
4
J. Torres
J. Torres
Citations:
13,149
h-index:
3
U. KamHou
U. KamHou
Citations:
13,124
h-index:
1
Kun Huang
Kun Huang
Citations:
13,172
h-index:
5
Kyle Huang
Kyle Huang
Citations:
13,149
h-index:
3
Lee Bell
Lee Bell
Citations:
13,149
h-index:
3
Lei Zhang
Lei Zhang
Citations:
13,151
h-index:
3
Licheng Yu
Licheng Yu
Citations:
13,644
h-index:
6
M. Bhatt
M. Bhatt
Citations:
13,380
h-index:
6
M. Lennie
M. Lennie
Citations:
13,246
h-index:
5
M. Groshev
M. Groshev
Citations:
13,200
h-index:
5
Maya Lathi
Maya Lathi
Citations:
13,150
h-index:
3
M. Seltzer
M. Seltzer
Citations:
23,676
h-index:
43
Michal Valko
Michal Valko
Building something new @ Stealth Startup & Inria & MVA - Ex: Llama @AIatMeta Gemini and BYOL @GoogleDeepMind
Citations:
28,340
h-index:
43
M. Patel
M. Patel
Citations:
13,159
h-index:
3
Mike Clark
Mike Clark
Citations:
13,205
h-index:
5
M. Macey
M. Macey
Citations:
13,250
h-index:
5
Mike Wang
Mike Wang
Citations:
13,150
h-index:
3
Mo Metanat
Mo Metanat
Citations:
13,203
h-index:
5
Mohammad Rastegari
Mohammad Rastegari
Iran University of Science and Technology
Citations:
14,407
h-index:
15
Nick Egebo
Nick Egebo
Citations:
13,155
h-index:
3
Ning Dong
Ning Dong
Citations:
13,202
h-index:
3
Ning Zhang
Ning Zhang
Citations:
13,243
h-index:
6
O. Hart
O. Hart
Citations:
13,151
h-index:
3
Paul Saab
Paul Saab
Citations:
13,203
h-index:
5
P. Rittner
P. Rittner
Citations:
13,167
h-index:
4
P. Yuvraj
P. Yuvraj
Citations:
13,324
h-index:
6
Qian Liang
Qian Liang
Citations:
13,274
h-index:
5
Rafi Ayub
Rafi Ayub
Citations:
13,344
h-index:
7
Raymond Li
Raymond Li
Citations:
13,231
h-index:
5
Rocky Wang
Rocky Wang
Citations:
13,206
h-index:
5
Russ Howes
Russ Howes
Citations:
21,978
h-index:
11
Sara Chugh
Sara Chugh
Citations:
13,206
h-index:
5
S. Sidorov
S. Sidorov
Citations:
13,125
h-index:
2
Sheng Feng
Sheng Feng
Citations:
13,213
h-index:
5
S. Shankar
S. Shankar
Citations:
13,136
h-index:
2
S. Gupta
S. Gupta
Citations:
14,175
h-index:
13
Sunny Virk
Sunny Virk
Citations:
13,178
h-index:
3
Tal Remez
Tal Remez
Citations:
18,470
h-index:
22
Tianhe Li
Tianhe Li
Citations:
13,615
h-index:
4
V. Poenaru
V. Poenaru
Citations:
13,197
h-index:
4
Wei Li
Wei Li
Citations:
13,191
h-index:
3
Sara Hunt
Sara Hunt
Citations:
13,276
h-index:
5
S. Pan
S. Pan
Citations:
13,757
h-index:
7
현대 인공지능(AI) 시스템은 파운데이션 모델(foundation models)에 의해 구동됩니다. 본 논문은 Llama 3라고 불리는 새로운 파운데이션 모델 세트를 제시합니다. 이는 다국어, 코딩, 추론 및 도구 사용을 기본적으로 지원하는 언어 모델군입니다. 가장 큰 모델은 4,050억(405B) 개의 파라미터와 최대 12만 8천(128K) 토큰의 컨텍스트 윈도우를 가진 Dense Transformer입니다. 본 논문은 Llama 3에 대한 광범위한 실증적 평가를 제시합니다. 평가 결과, Llama 3는 수많은 작업에서 GPT-4와 같은 선도적인 언어 모델과 대등한 품질을 제공하는 것으로 나타났습니다. 우리는 4,050억 파라미터 언어 모델의 사전 학습(pre-trained) 및 사후 학습(post-trained) 버전과 입출력 안전을 위한 Llama Guard 3 모델을 포함하여 Llama 3를 공개적으로 배포합니다. 또한 본 논문은 구성적(compositional) 접근 방식을 통해 이미지, 비디오 및 음성 기능을 Llama 3에 통합한 실험 결과를 제시합니다. 우리는 이 접근 방식이 이미지, 비디오 및 음성 인식 작업에서 최신 기술(state-of-the-art)과 경쟁력 있는 성능을 보임을 확인했습니다. 결과물로 나온 모델들은 아직 개발 중이므로 광범위하게 배포되지는 않습니다.
Original
Abstract
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
No Analysis Report Yet
This paper hasn't been analyzed by Gemini yet.