2407.21783
Jul 31, 2024
cs.AI
Llama 3 모델군(The Llama 3 Herd of Models)
The Llama 3 Herd of Models
Abhimanyu Dubey
Abhimanyu Dubey
Facebook AI Research
Citations:
17,143
h-index:
24
Amy Yang
Amy Yang
Citations:
14,996
h-index:
9
Angela Fan
Angela Fan
Citations:
15,117
h-index:
6
Aobo Yang
Aobo Yang
Citations:
14,795
h-index:
9
A. Mitra
A. Mitra
Citations:
14,686
h-index:
8
A. Korenev
A. Korenev
Citations:
31,296
h-index:
8
Arun Rao
Arun Rao
Citations:
14,666
h-index:
7
B. Biron
B. Biron
Citations:
15,174
h-index:
11
Binh Tang
Binh Tang
Citations:
14,872
h-index:
8
Chloe Bi
Chloe Bi
Citations:
14,775
h-index:
9
C. Touret
C. Touret
Citations:
14,949
h-index:
8
C. Wong
C. Wong
Citations:
14,939
h-index:
8
D. Song
D. Song
Citations:
15,030
h-index:
10
D. Pintz
D. Pintz
Citations:
14,676
h-index:
8
Diego Garcia-Olano
Diego Garcia-Olano
University of Texas at Austin
Citations:
14,896
h-index:
6
E. Smith
E. Smith
Citations:
14,821
h-index:
9
G. Nail
G. Nail
Citations:
14,738
h-index:
8
G. Mialon
G. Mialon
Citations:
16,667
h-index:
13
Hu Xu
Hu Xu
Citations:
14,696
h-index:
8
Jade Copet
Jade Copet
Citations:
22,008
h-index:
22
Jaewon Lee
Jaewon Lee
Citations:
14,642
h-index:
8
Jason Park
Jason Park
Citations:
14,666
h-index:
7
Jeet Shah
Jeet Shah
Citations:
14,624
h-index:
6
J. Billock
J. Billock
Citations:
14,631
h-index:
6
Jenny Hong
Jenny Hong
Citations:
14,624
h-index:
6
Jenya Lee
Jenya Lee
Citations:
31,424
h-index:
8
Jiawen Liu
Jiawen Liu
Citations:
14,643
h-index:
7
Jie Wang
Jie Wang
Citations:
14,656
h-index:
8
Jiecao Yu
Jiecao Yu
Citations:
14,780
h-index:
8
Joe Spisak
Joe Spisak
Citations:
16,241
h-index:
8
K. Upasani
K. Upasani
Citations:
15,828
h-index:
10
Keqian Li
Keqian Li
Citations:
29,166
h-index:
8
L. Maaten
L. Maaten
Citations:
89,688
h-index:
51
Liang Tan
Liang Tan
Citations:
15,030
h-index:
8
Lubo Malo
Lubo Malo
Citations:
14,608
h-index:
5
M. Muzzi
M. Muzzi
Citations:
14,609
h-index:
5
Mannat Singh
Mannat Singh
Facebook AI Research (FAIR)
Citations:
19,980
h-index:
14
Mike Lewis
Mike Lewis
Citations:
15,006
h-index:
9
Min Si
Min Si
Citations:
14,529
h-index:
4
Pengwei Li
Pengwei Li
Citations:
14,999
h-index:
8
Peter Weng
Peter Weng
Citations:
14,632
h-index:
6
P. Dubal
P. Dubal
Citations:
14,663
h-index:
7
Puxin Xu
Puxin Xu
Citations:
32,748
h-index:
9
Q. He
Q. He
Citations:
14,592
h-index:
6
Rohit Girdhar
Rohit Girdhar
Facebook AI Research
Citations:
29,168
h-index:
30
Ruan Silva
Ruan Silva
Citations:
31,318
h-index:
8
Rui Hou
Rui Hou
Citations:
15,264
h-index:
8
Rui Wang
Rui Wang
Citations:
14,919
h-index:
6
Sean Bell
Sean Bell
Citations:
15,123
h-index:
7
S. Kim
S. Kim
Citations:
14,500
h-index:
2
Sergey Edunov
Sergey Edunov
Facebook AI Research
Citations:
48,300
h-index:
19
Sheng Shen
Sheng Shen
University of California, Berkeley
Citations:
25,922
h-index:
34
Shun Zhang
Shun Zhang
Citations:
14,651
h-index:
8
S. Collot
S. Collot
Citations:
14,633
h-index:
6
Suchin Gururangan
Suchin Gururangan
Allen Institute for AI
Citations:
24,261
h-index:
24
T. Fowler
T. Fowler
Citations:
15,238
h-index:
8
Tong Xiao
Tong Xiao
Citations:
14,649
h-index:
7
Ujjwal Karn
Ujjwal Karn
Facebook
Citations:
14,699
h-index:
5
Weiwei Chu
Weiwei Chu
Citations:
14,654
h-index:
7
Wenyin Fu
Wenyin Fu
Citations:
31,427
h-index:
8
X. Tan
X. Tan
Citations:
14,963
h-index:
4
Xuchao Jia
Xuchao Jia
Citations:
14,547
h-index:
4
Yiqian Wen
Yiqian Wen
Citations:
14,956
h-index:
4
Yiwen Song
Yiwen Song
Citations:
17,218
h-index:
4
Yuchen Zhang
Yuchen Zhang
Meta AI
Citations:
31,372
h-index:
8
Yue Li
Yue Li
Citations:
14,759
h-index:
7
Yuning Mao
Yuning Mao
Citations:
15,758
h-index:
9
Abha Jain
Abha Jain
Citations:
14,623
h-index:
5
A. Kelsey
A. Kelsey
Citations:
14,621
h-index:
6
A. Menon
A. Menon
Citations:
14,693
h-index:
6
Anam Yunus
Anam Yunus
Citations:
14,586
h-index:
4
Andrei Lupu
Andrei Lupu
University of Oxford, Meta AI
Citations:
15,159
h-index:
11
A. Caples
A. Caples
Citations:
14,662
h-index:
6
Andrew Gu
Andrew Gu
Citations:
14,572
h-index:
3
Andrew Ho
Andrew Ho
Citations:
14,609
h-index:
5
Beau James
Beau James
Citations:
14,611
h-index:
5
Ben Maurer
Ben Maurer
Citations:
14,615
h-index:
5
Beth Loyd
Beth Loyd
Citations:
14,619
h-index:
6
Bing Liu
Bing Liu
Citations:
17,374
h-index:
9
Bo Wu
Bo Wu
Citations:
14,585
h-index:
5
B. Ni
B. Ni
Citations:
14,609
h-index:
6
Bram Wasti
Bram Wasti
Citations:
15,442
h-index:
7
Chao Zhou
Chao Zhou
Citations:
14,599
h-index:
5
Chester Hu
Chester Hu
Citations:
14,585
h-index:
4
Chris Cai
Chris Cai
Citations:
14,646
h-index:
7
D. Beaty
D. Beaty
Citations:
14,585
h-index:
4
David Xu
David Xu
Citations:
14,579
h-index:
5
Didem Foss
Didem Foss
Citations:
14,553
h-index:
3
Duc Le
Duc Le
Citations:
16,468
h-index:
23
Emily Hahn
Emily Hahn
Citations:
14,592
h-index:
5
Emily Wood
Emily Wood
Citations:
14,592
h-index:
5
E. Arcaute
E. Arcaute
Citations:
15,376
h-index:
7
Fei Sun
Fei Sun
Citations:
14,725
h-index:
7
F. Kreuk
F. Kreuk
Citations:
17,162
h-index:
19
Feng Tian
Feng Tian
Citations:
14,612
h-index:
6
Han Zou
Han Zou
Citations:
14,608
h-index:
6
Han Zha
Han Zha
Citations:
14,517
h-index:
3
Helen Suk
Helen Suk
Citations:
15,567
h-index:
3
Itai Gat
Itai Gat
Meta
Citations:
20,084
h-index:
21
Jeff Tang
Jeff Tang
Citations:
14,508
h-index:
3
J. Chan
J. Chan
Citations:
14,543
h-index:
4
Jenny Zhen
Jenny Zhen
Citations:
14,541
h-index:
4
J. Teboul
J. Teboul
Citations:
14,543
h-index:
4
J. Zhong
J. Zhong
Citations:
14,541
h-index:
4
Jian Jin
Jian Jin
Citations:
14,714
h-index:
7
J. Carvill
J. Carvill
Citations:
14,539
h-index:
3
J. McPhie
J. McPhie
Citations:
14,576
h-index:
4
J. Torres
J. Torres
Citations:
14,533
h-index:
3
U. KamHou
U. KamHou
Citations:
14,497
h-index:
2
Kun Huang
Kun Huang
Citations:
14,563
h-index:
6
Kyle Huang
Kyle Huang
Citations:
14,551
h-index:
4
Lee Bell
Lee Bell
Citations:
14,534
h-index:
3
Lei Zhang
Lei Zhang
Citations:
14,542
h-index:
4
Licheng Yu
Licheng Yu
Citations:
15,093
h-index:
7
M. Bhatt
M. Bhatt
Citations:
14,780
h-index:
6
M. Lennie
M. Lennie
Citations:
14,647
h-index:
6
M. Groshev
M. Groshev
Citations:
14,594
h-index:
5
M. Naumov
M. Naumov
Citations:
14,848
h-index:
7
Maya Lathi
Maya Lathi
Citations:
14,535
h-index:
3
M. Seltzer
M. Seltzer
Citations:
25,148
h-index:
44
Michal Valko
Michal Valko
Building something new @ Stealth Startup & Inria & MVA - Ex: Llama @AIatMeta Gemini and BYOL @GoogleDeepMind
Citations:
30,039
h-index:
44
M. Patel
M. Patel
Citations:
14,539
h-index:
4
Mike Clark
Mike Clark
Citations:
14,605
h-index:
6
M. Macey
M. Macey
Citations:
14,650
h-index:
6
Mike Wang
Mike Wang
Citations:
14,535
h-index:
3
Mo Metanat
Mo Metanat
Citations:
14,597
h-index:
5
Mohammad Rastegari
Mohammad Rastegari
Iran University of Science and Technology
Citations:
15,832
h-index:
15
Nick Egebo
Nick Egebo
Citations:
14,546
h-index:
4
Ning Dong
Ning Dong
Citations:
14,586
h-index:
3
Ning Zhang
Ning Zhang
Citations:
14,641
h-index:
6
O. Hart
O. Hart
Citations:
14,535
h-index:
3
Paul Saab
Paul Saab
Citations:
14,595
h-index:
5
P. Yuvraj
P. Yuvraj
Citations:
14,729
h-index:
7
Qian Liang
Qian Liang
Citations:
14,680
h-index:
6
Rafi Ayub
Rafi Ayub
Citations:
14,744
h-index:
7
Raymond Li
Raymond Li
Citations:
14,631
h-index:
6
Rocky Wang
Rocky Wang
Citations:
14,606
h-index:
6
Russ Howes
Russ Howes
Citations:
24,420
h-index:
11
Sara Chugh
Sara Chugh
Citations:
14,599
h-index:
5
S. Sidorov
S. Sidorov
Citations:
14,496
h-index:
2
Sheng Feng
Sheng Feng
Citations:
14,614
h-index:
6
S. Shankar
S. Shankar
Citations:
14,512
h-index:
2
S. Gupta
S. Gupta
Citations:
15,608
h-index:
14
Sunny Virk
Sunny Virk
Citations:
14,561
h-index:
3
Tal Remez
Tal Remez
Citations:
20,118
h-index:
22
Tianhe Li
Tianhe Li
Citations:
15,065
h-index:
5
V. Poenaru
V. Poenaru
Citations:
14,568
h-index:
4
V. Ivanov
V. Ivanov
Citations:
14,570
h-index:
4
Wei Li
Wei Li
Citations:
14,571
h-index:
3
Sara Hunt
Sara Hunt
Citations:
14,650
h-index:
5
S. Pan
S. Pan
Citations:
15,139
h-index:
7
Jeremy Fu
Jeremy Fu
Citations:
31,479
h-index:
9
현대 인공지능(AI) 시스템은 파운데이션 모델(foundation models)에 의해 구동됩니다. 본 논문은 Llama 3라고 불리는 새로운 파운데이션 모델 세트를 제시합니다. 이는 다국어, 코딩, 추론 및 도구 사용을 기본적으로 지원하는 언어 모델군입니다. 가장 큰 모델은 4,050억(405B) 개의 파라미터와 최대 12만 8천(128K) 토큰의 컨텍스트 윈도우를 가진 Dense Transformer입니다. 본 논문은 Llama 3에 대한 광범위한 실증적 평가를 제시합니다. 평가 결과, Llama 3는 수많은 작업에서 GPT-4와 같은 선도적인 언어 모델과 대등한 품질을 제공하는 것으로 나타났습니다. 우리는 4,050억 파라미터 언어 모델의 사전 학습(pre-trained) 및 사후 학습(post-trained) 버전과 입출력 안전을 위한 Llama Guard 3 모델을 포함하여 Llama 3를 공개적으로 배포합니다. 또한 본 논문은 구성적(compositional) 접근 방식을 통해 이미지, 비디오 및 음성 기능을 Llama 3에 통합한 실험 결과를 제시합니다. 우리는 이 접근 방식이 이미지, 비디오 및 음성 인식 작업에서 최신 기술(state-of-the-art)과 경쟁력 있는 성능을 보임을 확인했습니다. 결과물로 나온 모델들은 아직 개발 중이므로 광범위하게 배포되지는 않습니다.
Original
Abstract
Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.
Original PDF
{ copied = true; setTimeout(() => copied = false, 2000); })"
class="btn btn-square btn-sm btn-ghost text-base-content/50 hover:text-primary hover:bg-base-200 transition-colors mb-1"
:class="{ 'tooltip tooltip-success': copied, 'tooltip': !copied }"
:data-tip="copied ? '복사 완료!' : 'PDF 링크 복사'">
Copy PDF Link
No Analysis Report Yet
This paper hasn't been analyzed by Gemini yet.
Log in to request an AI analysis.