Rules, Heuristics, or LLMs? Lessons from Solving the Same Problem Twice

Not every problem needs an LLM. But at the same time some problems are asking for LLMs as the solution. So, when to choose which?

I ran into this while working on a subjective text transformation problem. It’s hard to specify and hard to test. That made it into a brilliant grey zone. When the answer to the regular regular question “can it be done without LLM” is “yes, but…”.

To understand the trade-offs, I built two solutions to the same problem. Both of them produce similar result, they just work in a very different way.

One is a “just code and math”: rule-based stochastic system using Markov chains, edit-distance mutations and so on. The other is a LoRA fine tuned LLM trained on the examples.

In this talk, I’ll share what I learned, so that you could build just one system, instead of two:

Where deterministic models offer better control Where LLMs produce more natural results How the results are different Maintenance cost

As the problems sits in a grey zone and hard to properly measure, I will show a result of blind comparison between rule-based output and LLM output to determine whether LLM solution was necessary or overkill.

This is not a tutorial or an AI demo. You’ll leave with a practical way to understand and decide when the problem is LLM-worthy and when to stick to the good old code and algorithms. It’s a case study on how over-engineering once on purpose can save future effort and resources.