Understanding LLM Architectures: Inside the Design of Modern Models

Large Language Models are often described as if each generation introduces an entirely new architecture. In practice, most modern LLMs still retain the transformer core, but their real progress comes from a series of targeted design changes around attention, positional handling, feed-forward computation, routing, and memory efficiency. This talk explains LLM architectures through the engineering tradeoffs that shaped modern models: why some attention mechanisms evolved for lower inference cost, how architectural choices influence long-context behavior, why sparse activation changed the economics of scale, and how these shifts affect real-world deployment. Rather than treating LLM architecture as a static diagram, this session presents it as a set of design decisions made in response to practical constraints in latency, memory, context length, and system efficiency. Attendees will leave with a clearer mental model of what remained stable, what changed, and why those changes matter.