GPT

GPT stands for Generative Pre-trained Transformer. OpenAI published GPT-1 in 2018, GPT-2 in 2019, GPT-3 in 2020, GPT-4 in 2023, GPT-5 in 2025. Each generation scaled parameters, training compute, and training data. Modern GPT models (GPT-5 class) are multi-modal, support reasoning, and serve as the foundation for ChatGPT's consumer product.

GPT architecture: decoder-only transformer with causal attention. Pretraining on internet-scale text with next-token prediction. Post-training includes SFT, RLHF (later DPO), and safety tuning. GPT-4 class models are mixture-of-experts · GPT-5 likely continues this direction. OpenAI ships multiple tiers per generation: flagship (GPT-5), efficient variants (GPT-5 mini, GPT-5 nano), and reasoning-specialized (o1, o3, o4-mini). Pricing varies dramatically · frontier tier $10-60 per million output tokens, efficient tier $0.10-2.

GPT-4 is believed to be ~1.8T total parameters with MoE sparsity (estimates vary · OpenAI has not disclosed). Training FLOPs estimated at 2e25 for GPT-4, possibly 5-10× for GPT-5. Compute scaling followed Chinchilla ratios until GPT-4; post-GPT-4 shifted to train-for-longer on same parameters. Post-training is OpenAI's proprietary differentiator · data mix, RLHF/DPO recipes, and safety tuning are closely held. Reasoning models in the o-series are RL-tuned on verifiable-reward tasks and use hidden CoT at inference. OpenAI serves GPT models via the API, ChatGPT, Azure OpenAI (Microsoft), and select enterprise partnerships.