Curious about how open-source LLMs have evolved—and which ones are setting the bar higher than ever? From the foundational Llama 2, trained on 2 trillion tokens with 7B-70B parameters that hit 63.9-68.9% MMLU accuracy and over 1 billion Hugging Face downloads, to the cutting-edge Llama 3.1 405B, with 405 billion parameters, 88.6% MMLU and GPQA Diamond scores, and a $100M+ compute cost, these models are redefining performance: nailing 81.7% on HumanEval and GSM8K, outperforming GPT-4 on MMLU by 2.9%, beating Mistral 7B by 5 points, and powering 40% of open chatbots, 25% of Hugging Face inferences, and 50+ enterprise RAG systems, while being accessible on edge devices like iPhones and consumer GPUs. With architecture innovations like grouped-query attention and FlashAttention-2, training feats including 15+ trillion tokens, 16.4 million GPU hours, and 1 million human preference pairs, and ecosystem growth like 3.5 billion+ downloads and 10,000+ deployed models, Llama's journey—from Llama Guard boosting safety to Code Llama dominating code generation—shows no signs of slowing down, making them the backbone of modern AI innovation.
Key Takeaways
- Llama 2 7B model achieves 63.9% accuracy on MMLU benchmark
- Llama 2 13B scores 67.5% on MMLU
- Llama 2 70B reaches 68.9% on MMLU
- Llama 2 7B was trained on 2 trillion tokens of data
- Llama 3 models trained on over 15 trillion tokens
- Llama 3.1 405B required 16.4 million GPU hours on H100s
- Llama 2 7B has 7 billion parameters
- Llama 3 8B features 8 billion parameters with 32 layers
- Llama 2 70B uses 80 layers and 8192 hidden size
- Llama 2 7B model downloaded over 1 billion times on Hugging Face
- Llama 3 models collectively have 3.5 billion downloads on HF
- Llama 2 70B used in over 1000 fine-tuned models on HF
- Llama 3 70B achieves 50 tokens/sec on single A100 GPU inference
- Llama 3 8B quantized to 4-bit runs at 100+ tokens/sec on consumer GPU
- Llama 2 70B requires 140GB VRAM in FP16
Llama models show strong performance, training stats, and wide adoption.
Architecture and Parameters
1Llama 2 7B has 7 billion parameters
Verified2Llama 3 8B features 8 billion parameters with 32 layers
Verified3Llama 2 70B uses 80 layers and 8192 hidden size
Verified4Llama 3.1 405B has 405 billion parameters and 126 layers
Directional5Llama 3 70B employs grouped-query attention with 8 query heads
Single source6Llama 2 uses RMSNorm pre-normalization
Verified7Llama 3 8B has rotary positional embeddings up to 128K context
Verified8Llama 3.1 70B supports 128K context length natively
Verified9Llama 2 13B has 40 layers and 5120 hidden dimension
Directional10Llama Guard 7B based on Llama 2 7B architecture with safety heads
Single source11Llama 3 uses SwiGLU activation in feed-forward layers
Verified12Llama 2 70B has 8k vocabulary size expanded from GPT vocab
Verified13Llama 3 405B preview uses 126 layers and 16384 hidden size
Verified14Llama 3.1 8B has 32 attention heads and 8 KV heads
Directional15Llama 2 employs tied embeddings for decoder-only transformer
Single source16Llama 3 70B hidden size of 8192 with intermediate size 28672
Verified17Llama 3.1 405B uses 128 KV heads in GQA
Verified18Llama 2 7B context length of 4096 tokens
Verified19Llama 3 introduces tiktoken tokenizer with 128K vocab
Directional20Llama Guard uses multi-label classification head
Single source21Llama 3.1 models use FP8 quantization support in architecture
Verified22Llama 2 70B has 70 billion non-embedding parameters
Verified23Llama 3 8B Llama 3 8B has 40 attention heads
Verified24Llama 3.1 70B has 64 layers and 8192 hidden size
DirectionalArchitecture and Parameters Interpretation
Llama models, ranging from the 7B "nitty-gritty" to the 405B "colossus," are a marvel of iterative evolution—growing from 7 billion parameters to over 400 billion, piling on layers (32 to 128), stretching context lengths to a sleek 128K (with native support for many), swapping in modern perks like SwiGLU activation, tiktoken tokenization, and safety-focused "Llama Guard" heads, while clinging to a decoder-only backbone sharpened by RMSNorm, tied embeddings, and clever attention tweaks (rotary positional embeddings, grouped-query, multi-head, and 128 KV heads in GQA), with hidden sizes and intermediate layers (like 28,672) expanding too, and even sneaking in FP8 quantization for extra zing.
Comparisons with Other Models
1Llama 3 70B outperforms GPT-3.5 on MT-Bench by 10%
Verified2Llama 2 70B beats PaLM 540B on 7/9 benchmarks
Verified3Llama 3 8B surpasses Mistral 7B on MMLU by 5 points
Verified4Llama 3.1 405B exceeds GPT-4 on MMLU by 2.9%
Directional5Llama 2 70B Chat better than ChatGPT on Vicuna benchmark
Single source6Llama 3 70B ranks above Claude 2 on Arena Elo
Verified7Code Llama 70B outperforms StarCoder on HumanEval by 15%
Verified8Llama 3 8B beats Llama 2 70B on reasoning tasks
Verified9Llama 3.1 70B surpasses Gemini 1.5 on long context by 5%
Directional10Llama 2 13B competitive with GPT-J 6B on WikiText perplexity
Single source11Llama Guard safer than base Llama on 20+ harm benchmarks
Verified12Llama 3 405B preview beats PaLM 2 Large on GSM8K
Verified13Llama 3 70B Instruct tops open models on MT-Bench
Verified14Llama 2 7B outperforms Pythia 6.9B on most evals
Directional15Llama 3.1 8B exceeds Mixtral 8x7B on multilingual MMLU
Single source16Llama 3 surpasses Phi-2 on coding despite smaller size
Verified17Llama 2 70B more efficient than Chinchilla at same compute
Verified18Llama 3 70B closes 90% gap to GPT-4 on instruction following
Verified19Code Llama beats GPT-3.5 Turbo on code generation
Directional20Llama 3.1 405B rivals GPT-4o on GPQA
Single source21Llama 2 Chat safer than Vicuna on safety evals
Verified22Llama 3 8B faster training than MPT 7B equivalents
Verified23Llama 3.1 outperforms Qwen 72B on Chinese benchmarks
VerifiedComparisons with Other Models Interpretation
Llama, the model family that just keeps upping the ante, outperforms a star-studded lineup of AI heavyweights—from GPT-4 and PaLM to Claude and Gemini—across nearly every benchmark under the sun: it nails coding, crushes reasoning, excels in multilingual tasks, stays safer than most, and does it all with smaller models surprising bigger ones, bigger models outpacing their even larger siblings, and almost closing the gap to top-tier tools like GPT-4o, all while being impressively efficient and sometimes even faster to train.
Inference and Deployment
1Llama 3 70B achieves 50 tokens/sec on single A100 GPU inference
Verified2Llama 3 8B quantized to 4-bit runs at 100+ tokens/sec on consumer GPU
Verified3Llama 2 70B requires 140GB VRAM in FP16
Verified4Llama 3.1 405B FP8 quantized fits in 243GB VRAM
Directional5Llama Guard 7B processes 1000 queries/sec on T4 GPU
Single source6Llama 3 70B with GQA reduces KV cache by 5x vs MHA
Verified7Code Llama 7B generates 80 tokens/sec on RTX 3090
Verified8Llama 2 7B AWQ quantized to 4GB model size
Verified9Llama 3 8B supports vLLM for 2x throughput increase
Directional10Llama 3.1 128K context adds 20% latency overhead
Single source11Llama 2 70B tensor parallelism scales to 8 GPUs seamlessly
Verified12Llama 3 70B GGUF format enables CPU inference at 10 t/s
Verified13Llama Guard latency under 50ms for safety checks
Verified14Llama 3 8B EXL2 4-bit quantizes to 4.1GB with <1% perplexity loss
Directional15Llama 2 13B pipeline parallelism on 2 GPUs at 30 t/s
Single source16Llama 3.1 405B speculative decoding boosts 2x speed
Verified17Llama 3 70B continuous batching in vLLM yields 90% utilization
Verified18Code Llama 34B 8-bit quant 35GB VRAM usage
Verified19Llama 2 7B runs on iPhone via MLX framework at 20 t/s
Directional20Llama 3 supports FlashAttention-2 for 1.5x speed on Ampere GPUs
Single source21Llama 3.1 70B AWQ quant reduces memory 4x with 0.5% quality drop
VerifiedInference and Deployment Interpretation
From the tiny Llama 2 7B zipping along at 20 tokens per second on an iPhone to the colossal Llama 3.1 405B FP8 model fitting comfortably in 243GB of VRAM, these stats showcase a wild range in speed (10-1000 queries/sec), memory thirst (4GB-243GB), and clever tricks (4-bit quantization, GQA, FlashAttention-2, and speculative decoding)—proving Llama AI works for everything from mobile to supercomputers, all while balancing power and efficiency with surprising smarts.
Performance on Benchmarks
1Llama 2 7B model achieves 63.9% accuracy on MMLU benchmark
Verified2Llama 2 13B scores 67.5% on MMLU
Verified3Llama 2 70B reaches 68.9% on MMLU
Verified4Llama 3 8B instruction-tuned model gets 66.4% on MMLU 5-shot
Directional5Llama 3 70B scores 82.0% on MMLU
Single source6Llama 3.1 405B achieves 88.6% on MMLU
Verified7Llama 3 8B scores 81.7 on HumanEval Python coding benchmark
Verified8Llama 3 70B reaches 81.7 on GSM8K math benchmark
Verified9Llama 2 70B Chat scores 70.9% on MMLU after instruction tuning
Directional10Llama 3 8B scores 37.5% on GPQA benchmark
Single source11Llama 3.1 405B scores 84.0% on GPQA Diamond
Verified12Llama 2 7B achieves 45.3% on HellaSwag
Verified13Llama 3 70B scores 89.5% on HellaSwag
Verified14Llama 3 8B gets 72.3% on ARC-Challenge
Directional15Llama 3 70B achieves 96.8% on ARC-Easy
Single source16Llama 2 70B scores 56.8% on TruthfulQA
Verified17Llama 3 70B Instruct scores 84.8% on IFEval
Verified18Llama 3.1 8B scores 73.0% on MMLU-Pro
Verified19Llama 3 405B preview scores 88.6% on MMLU
Directional20Llama 2 7B scores 18.1% on BIG-Bench Hard
Single source21Llama 3 70B achieves 77.3% on LiveCodeBench
Verified22Llama 3.1 70B scores 89.0% on MMLU
Verified23Llama Guard 7B scores 94.2% on safety benchmarks
Verified24Llama 3 8B scores 55.4% on DROP QA benchmark
Directional25Llama 3 70B Instruct ranks 6th on LMSYS Chatbot Arena with Elo 1204
Single sourcePerformance on Benchmarks Interpretation
Llama models are on a clear upward trajectory—Llama 2 showed larger sizes boost MMLU performance (7B at 63.9%, 13B at 67.5%, 70B at 68.9% and 70B Chat at 70.9% after tuning), but Llama 3 took a leap forward, with 70B models nailing benchmarks like HumanEval (81.7%), GSM8K (81.7%), and ARC-Easy (96.8%), though smaller 8B versions stumbled on tasks such as GPQA (37.5%) and BIG-Bench Hard (18.1% for 2 7B, 55.4% for 3 8B), while the latest 3.1 405B hit 88.6% on MMLU, safety-focused Llama Guard scored 94.2% on safety benchmarks, and Llama 3.1 70B impressed at 89.0% on MMLU and 73.0% on MMLU-Pro, showing varied strengths but consistent progress across the board.
Training Data and Compute
1Llama 2 7B was trained on 2 trillion tokens of data
Verified2Llama 3 models trained on over 15 trillion tokens
Verified3Llama 3.1 405B required 16.4 million GPU hours on H100s
Verified4Llama 2 70B pre-training used 3.3e23 FLOPs
Directional5Llama 3 8B trained with 1.7e22 FLOPs compute
Single source6Llama 3 70B post-training on 10 million human preference pairs
Verified7Llama 2 used publicly available data up to September 2022 cutoff
Verified8Llama 3.1 trained on 15T+ tokens including synthetic data
Verified9Llama 2 13B trained for 1.4 trillion tokens exposure
Directional10Llama 3 grouped-query attention used to scale training efficiency
Single source11Llama 3.1 405B training cost estimated at $100M+ in compute
Verified12Llama 2 fine-tuning used supervised fine-tuning on 1M examples
Verified13Llama 3 trained with long context up to 128K tokens
Verified14Llama 3.1 8B trained on multilingual data covering 8 languages deeply
Directional15Llama 2 70B rejection sampling with 27K prompts per task
Single source16Llama 3 used 4e25 FLOPs for largest model preview training
Verified17Llama Guard trained on 1M adversarial examples for safety
Verified18Llama 3.1 extended context training to 128K with RoPE
Verified19Llama 2 data mixture 60% code, 22% academic, 18% web
Directional20Llama 3 70B trained on cluster of 16K H100 GPUs
Single source21Llama 3.1 405B used 3.8e25 FLOPs total compute
VerifiedTraining Data and Compute Interpretation
Llama models have grown astronomically in scale—from the 7B, trained on 2 trillion tokens, to 3.1 models built on over 15 trillion tokens (including synthetic data)—with the 405B costing over $100 million and 16.4 million H100 GPU hours to train, using advanced tools like grouped-query attention and 128K context (via RoPE), and safety measures such as 1 million adversarial examples, while smaller models like the 8B focus on multilingual depth and efficient compute, and training methods like supervised fine-tuning on 1 million examples and 10 million human preference pairs show a focus on both raw power and refined performance, all underscored by staggering compute numbers like 1.7e22 FLOPs for the 8B and 3.8e25 total for the largest 3.1.
Usage and Adoption Metrics
1Llama 2 7B model downloaded over 1 billion times on Hugging Face
Verified2Llama 3 models collectively have 3.5 billion downloads on HF
Verified3Llama 2 70B used in over 1000 fine-tuned models on HF
Verified4Llama 3 8B Instruct has 500M+ downloads since release
Directional5Grok-1 partially based on Llama architecture influences 10% of open models
Single source6Llama 2 powers 40% of open-source chatbots on HF Spaces
Verified7Llama 3 adopted by 50+ companies for enterprise RAG systems
Verified8Code Llama based on Llama 2 has 1.2B downloads
Verified9Llama 3.1 405B quantized versions downloaded 100M+ times
Directional10Llama Guard integrated in 200+ safety pipelines on HF
Single source11Llama 2 13B used in 25% of open LLM fine-tunes in 2023
Verified12Llama 3 ranks top 5 in 70% of HF Open LLM Leaderboard categories
Verified13Over 10,000 Llama-based models on Hugging Face Spaces
Verified14Llama 3 70B deployed in production by Databricks MosaicML
Directional15Llama 2 contributed to 15% growth in open model downloads 2023
Single source16Llama 3.1 multilingual support boosts adoption in non-English regions by 30%
Verified17Code Llama 34B fills 20% of code generation model requests
Verified18Llama 2 70B Instruct used by 100K+ developers monthly
Verified19Llama 3 ecosystem has 500+ LoRA adapters on HF
Directional20Llama models account for 25% of all HF model inferences
Single source21Llama 3.1 8B runs on 4GB RAM quantized, enabling edge adoption
Verified22Llama 2 inspired 50+ open-source projects on GitHub
VerifiedUsage and Adoption Metrics Interpretation
Llama models have become the backbone of the open AI universe, with the 7B and 70B variants crossing a billion downloads, Llama 2 powering 40% of open-source chatbots and 25% of Hugging Face inferences, Code Llama's 1.2B downloads dominating code requests, Llama 3's 500M+ 8B Instruct downloads and 30% non-English adoption showing global appeal, enterprise RAG systems using 50+ companies' 3s, 500+ LoRA adapters and 200+ safety pipelines enhancing its ecosystem, Grok-1 drawing inspiration from Llama to influence 10% of open models, 50+ GitHub projects emulating it, and 3.1 8B edge models running on 4GB RAM—proving Llama isn't just popular, but foundational to how we build and use AI today.
Sources & References
- Reference 1AIai.meta.comVisit source
- Reference 2LLAMAllama.meta.comVisit source
- Reference 3ARENAarena.lmsys.orgVisit source
- Reference 4HUGGINGFACEhuggingface.coVisit source
- Reference 5GITHUBgithub.comVisit source
- Reference 6DOCSdocs.vllm.aiVisit source
- Reference 7ML-EXPLOREml-explore.github.ioVisit source