> Markdown version of [/videos/692-making-sense-of-programming-languages?t=463](https://www.wearedevelopers.com/videos/692-making-sense-of-programming-languages?t=463). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Making Sense of Programming Languages Understanding compilation pipelines isn't just academic theory. Master abstract syntax trees to ditch regex hacks and build your own custom linters, parsers, and dynamic domain-specific languages. - **Speakers:** Aleksandra Sikora - **Event:** World Congress 2023 - **Published:** September 27, 2023 - **Duration:** 31:50 - **URL:** https://www.wearedevelopers.com/videos/692-making-sense-of-programming-languages ## Summary Understanding the inner workings of programming languages is often perceived as intimidating, but learning the compilation and execution pipeline heavily pays off for real-world development. Grasping how abstract syntax trees (ASTs) operate enables developers to build powerful custom linting rules, enhance IDE functionality for replacing boilerplate, or design custom domain-specific languages (DSLs) for complex business logic. Because programming languages utilize "context-free" grammars to allow recursive rules, attempting to brute-force code transformation with regular expressions is ineffective, making dedicated parsers essential. The categorization of a language as purely "compiled" or "interpreted" is a common misconception; execution behavior depends entirely on the language's implementation. For example, JavaScript engines like V8 employ both an Ignition compiler to generate optimized bytecode and an interpreter to execute it. Across implementations, the execution road consistently begins with a scanner generating a stream of tokens and a parser structuring those tokens into a contextual syntactic tree. From there, the implementation might feed into a tree-walk interpreter or pass through semantic analysis and compiler optimizations to produce machine executables. Demonstrating these concepts in practice, developers can leverage tools like TypeScript Parsec to rapidly construct custom tokenizers and parsers. By defining specific token enums and mapping recursive expression rules—such as functions or variable declarations—builders construct a working syntax tree environment capable of dynamically evaluating scopes. Extending an overarching language is simply a matter of explicitly introducing a new lexical token, establishing its pattern within the parser's program sequence, and instructing the interpreter's switch-statement on how to evaluate and return its corresponding result. **Keywords:** abstract syntax tree, custom linting rules, domain-specific language, context-free grammar, language tokenization, AST node manipulation, V8 engine architecture, bytecode interpretation, compiler optimization, tree-walk interpreter, typescript-parsec, lexical scanner, recursive syntax definition, semantic evaluation, programming syntax parser ## Chapters 1. **Introduction to programming language internal workings** (00:04) — The fundamental concepts of programming language construction and why developers benefit from understanding them. 1. **Practical applications of language parsing knowledge** (01:40) — How understanding language structures enables custom linting rules, advanced editor automations, and domain-specific languages. 1. **Defining programming languages using context-free grammars** (04:30) — The role of regular and context-free grammars in defining valid syntax and language rules. 1. **Clarifying compiled versus interpreted language implementations** (07:43) — Why tools are categorized by their implementation rather than strictly labeled as interpreted or compiled languages. 1. **Tracing the execution pipeline from tokens to code** (10:48) — The complete pipeline transforming source code into evaluated results via scanning, parsing, and execution. 1. **Tokenizing input source code with a custom lexer** (13:49) — Implementing a custom lexer using TypeScript utilities to convert code strings into categorized tokens. 1. **Defining parser rules for an abstract syntax tree** (17:13) — Constructing an abstract syntax tree by mapping token sequences to structured grammar rules. 1. **Structuring a tree-walk interpreter for code evaluation** (19:53) — Building a minimal tree-walk interpreter that recursively evaluates parsed abstract syntax tree nodes. 1. **Extending language syntax in the lexer and parser** (23:03) — Modifying the custom language's lexer and parser to recognize and type a new print expression operator. 1. **Executing new syntax within an updated language interpreter** (29:16) — Updating the interpreter logic to evaluate and successfully run the newly added print statement syntax. ## Related Moments - [Demonstrating a custom language grammar with mini logo](https://www.wearedevelopers.com/videos/762-langium-design-your-own-language-in-node-js-and-vs-code) (from "Langium: Design your own language in Node.js and VS Code") - [Managing compilation performance and implementation specifics of advanced typing](https://www.wearedevelopers.com/videos/455-where-we-re-going-we-don-t-need-javascript-programming-with-type-annotations) (from "Where we're going we don't need JavaScript - Programming with Type Annotations") - [Abstractions and the push towards compiling natural language](https://www.wearedevelopers.com/videos/1909-wearedevelopers-live-back-to-code100) (from "WeAreDevelopers LIVE - Back to CODE100") - [Understanding how natural and programming languages evolve](https://www.wearedevelopers.com/videos/701-vikings-language-the-speech-of-the-king-vasa-or-today-s-swedish-text-classification-with-ml-net) (from "Vikings language, the speech of the king Vasa or today's Swedish? Text classification with ML.NET.") - [Transforming abstract syntax trees using Babel parsers and generators](https://www.wearedevelopers.com/videos/305-let-s-build-a-vs-code-extension-for-automated-refactorings) (from "Let's build a VS Code extension for automated refactorings") - [Leveraging abstract syntax trees for reliable code transformation](https://www.wearedevelopers.com/videos/305-let-s-build-a-vs-code-extension-for-automated-refactorings) (from "Let's build a VS Code extension for automated refactorings") ## Related Articles - [Dev Digest 136 - No JS(on) of mine](https://www.wearedevelopers.com/magazine/482-dev-digest-136-no-js-on-of-mine) - [Top in-demand programming languages to learn in 2021](https://www.wearedevelopers.com/magazine/30-top-in-demand-programming-languages-to-learn-in-2021) - [4 reasons why you should learn Rust in 2021 – and maybe even have fun doing it](https://www.wearedevelopers.com/magazine/35-4-reasons-why-you-should-learn-rust-in-2021-and-maybe-even-have-fun-doing-it) - [Dev Digest 124 - None like it hot](https://www.wearedevelopers.com/magazine/460-dev-digest-124-none-like-it-hot) ## Related Jobs - [Senior Software Engineer](https://www.wearedevelopers.com/jobs/ext/15942-senior-software-engineer) at **GitHub** - [Principal Software Engineer, Enterprise AI Platform](https://www.wearedevelopers.com/jobs/ext/1467292-principal-software-engineer-enterprise-ai-platform) at **GitHub** - [Staff Software Engineer, Copilot Experiences](https://www.wearedevelopers.com/jobs/ext/164361-staff-software-engineer-copilot-experiences) at **GitHub** - [Staff Developer Advocate, GitHub Security Lab](https://www.wearedevelopers.com/jobs/ext/1921051-staff-developer-advocate-github-security-lab) at **GitHub** - [Backend Engineer, Python (Web3)](https://www.wearedevelopers.com/jobs/ext/1934515-backend-engineer-python-web3) at **Bitpanda** - [Senior Software Engineer, Client Apps Platform](https://www.wearedevelopers.com/jobs/ext/1773893-senior-software-engineer-client-apps-platform) at **GitHub**