$devvkit resources --research llm-architecture-guide:-decoder-only-vs-encoder-decoder
LLM Architecture Guide: Decoder-Only vs Encoder-Decoder
3min[llm][architecture][deep-learning]
TL;DR: Most modern LLMs use decoder-only architecture (GPT, Llama, Claude) rather than encoder-decoder (T5, BART), trading bidirectional context for efficient autoregressive generation.
Key findings
- •Decoder-only models use causal (masked) self-attention where each token can only attend to previous tokens, making generation a sequential process.
- •Encoder-decoder models process input with bidirectional attention (encoder) and generate with causal attention (decoder), useful for translation and summarization.
- •Scaling laws show decoder-only models generalize better at large scales, which is why GPT-4, Claude, and Llama all follow this architecture.
- •KV caching is the key optimization for decoder-only inference, storing computed key/value matrices to avoid recomputation.
Why this matters for developers
Choosing the right architecture affects API costs, latency, and task suitability. Decoder-only models excel at open-ended generation. Encoder-decoder models excel at tasks requiring full input understanding.