> Markdown version of [/videos/100112-rules-heuristics-or-llms-lessons-from-solving-the-same-problem-twice](https://www.wearedevelopers.com/videos/100112-rules-heuristics-or-llms-lessons-from-solving-the-same-problem-twice). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Rules, Heuristics, or LLMs? Lessons from Solving the Same Problem Twice

One engineering team built a drunk text generator twice. They pitted traditional heuristics against a fine-tuned LLM. Discover which approach truly minimizes hidden technical debt.

- **Speakers:** [Artur Naumenko](https://www.wearedevelopers.com/@artur-naumenko)
- **Event:** World Congress 2026 Europe
- **Published:** July 9, 2026
- **Duration:** 22:16
- **URL:** https://www.wearedevelopers.com/videos/100112-rules-heuristics-or-llms-lessons-from-solving-the-same-problem-twice

## Summary

When facing subjective, hard-to-measure output requirements, engineering teams often struggle to choose between deterministic code and large language models (LLMs). This case study explores a "gray zone" text transformation problem—intentionally corrupting messages to appear naturally "drunk"—to evaluate the specific trade-offs of using traditional stochastic algorithms versus a fine-tuned AI model. By building two entirely distinct systems to solve the exact same abstract problem, evaluating them side-by-side reveals crucial insights into when to stick with algorithmic rules and when reaching for an LLM actually secures long-term engineering efficiency.

The initial deterministic approach utilizes a Java heuristic engine leveraging Markov chains, Levenshtein edit distance, and QWERTY keyboard adjacency models. While highly efficient to run and completely transparent for strict debugging routines, tuning its myriad of separate probability dials manually proved unexpectedly fragile, periodically resulting in over-corrupted noise or completely untouched outputs. Conversely, a parallel solution leveraging a Mistral 7B model adapted using LoRA (Low-Rank Adaptation) successfully bypassed complex manual parameter tuning. Although computationally heavier and inevitably opaque when edge cases misfire, the LLM reliably absorbed nuanced behaviors directly from its training distribution. Paradoxically, over-engineering the rules-based system initially provided the necessary procedural engine to dynamically generate the 22,000 synthetic data pairs required to train the functional language model.

For architects navigating architectural system dilemmas, the ideal path ultimately rests on domain definition and essential control boundaries. If an application mandates strict auditability, definitive boundaries, or the ability to explicitly override isolated behaviors, traditional code syntax remains overwhelmingly superior. However, if an input space is so vast it dictates an endless loop of conditional statements, or if the benchmarks for "good" textual output are heavily subjective, LLMs dramatically reduce operational maintenance capability. While bespoke heuristic infrastructures minimize immediate runtime cloud costs, deploying specialized LLMs for fuzzy subjective transformations routinely minimizes the hidden technical debt of managing highly fragile algorithms.

**Keywords:** rule-based heuristics, stochastic software systems, lora fine-tuning, mistral 7b deployment, synthetic training data generation, deterministic programming boundaries, markov chain generation, levenshtein edit distance, subjective natural language processing, algorithm parameter tuning, machine learning denoising, ai engineering trade-offs, heuristic edge cases, computational overhead balance

## Chapters

1. **Solving the same problem twice with code and LLMs** (00:11) — Overengineering a project on purpose reveals the differences between rule-based logic and fine-tuned language models.
1. **Identifying the grey zone between heuristics and language models** (01:10) — Problems with subjective outcomes or infinite edge cases require choosing between probabilistic heuristics and data-driven models.
1. **Corrupting text as a test case for subjective metrics** (05:18) — Generating realistic typos provides a perfect test case because measuring text corruption relies purely on subjective observation.
1. **Using rule-based heuristics to simulate realistic typing errors** (07:12) — Combining Markov chains and edit distances creates a configurable but highly complex programmatic engine for text corruption.
1. **Training a language model with synthetically generated text pairs** (08:45) — Fine-tuning a lightweight model with LoRA requires thousands of examples generated by the initial heuristic engine.
1. **Evaluating output quality between algorithmic tuning and machine learning** (10:13) — While both methods produce believable results, rule-based systems often output extreme noise compared to the natural averaging of models.
1. **Balancing execution costs and maintenance flexibility in production systems** (11:39) — Algorithmic approaches offer cheap execution and deep debuggability while machine learning systems demand higher computational resources but handle complexity automatically.
1. **Four questions for deciding between code and language models** (16:38) — Analyzing output metrics, debugging requirements, input space width, and bounds definition determines the appropriate technical solution.
1. **Why handbuilt systems often require more effort than models** (18:01) — Hand-tuning complex probabilistic systems frequently proves more difficult than allowing a machine learning model to learn the desired behavior.
1. **Hardware requirements and tuning heuristics with machine learning models** (19:53) — Running lightweight models relies on consumer hardware while attempting to tune algorithmic parameters automatically introduces significant system fragility.

## Related Moments

- [Introducing LLMs as judges for automated testing](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems) (from "Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems")
- [Natural language programming and prompt engineering](https://www.wearedevelopers.com/videos/1289-wad-live-22-01-2025-exploring-ai-web-development-and-accessibility-in-tech-with-stefan-judis) (from "WAD Live 22/01/2025: Exploring AI, Web Development, and Accessibility in Tech with Stefan Judis")
- [Using LLMs to reverse engineer undocumented legacy code](https://www.wearedevelopers.com/videos/100331-fighting-the-next-wave-of-cybercrime) (from "Fighting the Next Wave of Cybercrime")
- [Deciding if large language models are actually necessary](https://www.wearedevelopers.com/videos/1156-lessons-learned-building-a-genai-powered-app) (from "Lessons Learned Building a GenAI Powered App")
- [Evaluating statistical models and the bitter lesson](https://www.wearedevelopers.com/videos/100343-the-llm-evolution-from-sequence-imitation-to-verifiable-reasoning) (from "The LLM Evolution: From Sequence Imitation to Verifiable Reasoning")
- [Balancing artificial intelligence tools with foundational software engineering skills](https://www.wearedevelopers.com/videos/913-tech-with-tim-at-wearedevelopers-world-congress-2024) (from "Tech with Tim at WeAreDevelopers World Congress 2024")

## Related Articles

- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [WWC24 Talk - Scott Hanselman - AI: Superhero or Supervillain?](https://www.wearedevelopers.com/magazine/469-wwc24-talk-scott-hanselman-ai-superhero-or-supervillain)
- [How to Use Generative AI to Accelerate Learning to Code](https://www.wearedevelopers.com/magazine/530-how-to-use-generative-ai-to-accelerate-learning-to-code)

## Related Jobs

- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/588393-machine-learning-engineer) at **Twilio**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1355348-machine-learning-engineer) at **TWILIO**
- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Senior AI Agent Software Engineer (Go, Python) (m/f/x)](https://www.wearedevelopers.com/jobs/48277-senior-ai-agent-software-engineer-go-python-m-f-x) at **Dynatrace**
- [Staff, Machine Learning Engineer (L4)](https://www.wearedevelopers.com/jobs/ext/1202639-staff-machine-learning-engineer-l4) at **Twilio**
- [Machine Learning Engineer](https://www.wearedevelopers.com/jobs/ext/1597388-machine-learning-engineer) at **ZEISS Group**