> Markdown version of [/magazine/803-scaling-e-commerce-search-3-million-products-1-500-rps-under-200ms](https://www.wearedevelopers.com/magazine/803-scaling-e-commerce-search-3-million-products-1-500-rps-under-200ms). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Scaling E-commerce Search: 3 Million Products, 1,500 RPS, Under 200ms **By:** Yuri Romanenkov, [Daniel Cranney](https://www.wearedevelopers.com/@daniel-cranney) **Published:** November 18, 2026 Leo Tolstoy opens _Anna Karenina_ with a famous observation: “All happy families are alike; each unhappy family is unhappy in its own way.” Large-scale search systems behave similarly, and in this article we’ll explore how. Healthy search infrastructure is usually predictable: stable latency, manageable indexing pipelines and consistent relevance under load. But once search systems begin failing at scale, every architecture develops its own unique failure mode. Some systems collapse under indexing pressure. Others suffer from oversized documents, expensive relevance calculations or uncontrolled fuzzy matching. In many cases, Elasticsearch itself is not actually the bottleneck. The real problems are introduced much earlier — in document structure, query preprocessing and the growing number of “smart” search features added over time. ![A chart showing the complexity of a search request](https://wearedevelopers.imgix.net//magazine/articles/803/images/content/bZcpsDYQkU1cBOiqOK1u-1794237476.png?w=984&auto=compress,format) _Search complexity amplification: how fuzzy matching, synonym expansion and token permutations increase candidate documents, scoring operations and latency variance under production load_ That was the situation we faced while working with a marketplace-scale search system handling millions of products and roughly 1500 full-text search requests per second. Most queries still completed within the target SLA of 200ms, but some suddenly exceeded 500ms. Indexing delays stretched toward several hours, and search quality improvements increasingly came at the cost of system stability. The interesting part was that Elasticsearch itself was not really the root problem. Most performance issues had been created long before the query ever reached the cluster. # The First Bottleneck: Index Structure One of the earliest architectural mistakes was to treat Elasticsearch as a generalised product database rather than a specialised search engine. Initially, the index contained deeply nested documents with multiple levels of hierarchy. Product documents also stored city-specific information, such as availability and regional stock levels, directly inside the search index itself. At first, this approach seemed convenient. A single query could theoretically return all the information needed to create a product page. However, as the catalogue grew to include millions of products, this flexibility became increasingly costly. Documents became larger, indexing slowed down, and relevance scoring required significantly more computation. The complexity of queries also increased over time because the search layer gradually absorbed more and more business logic that was not directly related to the search itself. The turning point came when we started simplifying the index rather than enriching it further. We realised that the search document should ideally perform only one task: finding relevant products as quickly as possible. Pricing, discounts, stock information, and regional availability could often be handled later in the request flow, or even moved into separate pipelines. ![A chart showing how complexity can be removed from a query at the point when the request is made](https://wearedevelopers.imgix.net//magazine/articles/803/images/content/uUSr5UPiYX6wJJNRScRP-1794237477.png?w=984&auto=compress,format) _Elasticsearch scales more predictably when it acts as a dedicated search layer rather than a generalized product database containing pricing,_ _stock and business logic_ This decision introduced trade-offs elsewhere in the architecture, but it made the search layer much more predictable. Flattening document structures and removing unnecessary nested data became one of the biggest contributors to stable latency under load. # Smart search features became the main source of latency The next problem arose from improvements to search quality themselves. Modern Elasticsearch setups can support fuzzy matching, synonym expansion, prefix searches, and token-order-independent queries relatively easily. However, the danger is that these features often appear inexpensive in isolation, but become extremely expensive when used in combination. One particularly costly feature is word-order-independent search. Users naturally expect queries such as ‘red socks’ and ‘socks red’ to produce the same results, but supporting this flexibility increases query complexity much faster than many teams initially anticipate ![A diagram comparing a pure full-text search term accessing 2.4 million documents and a structured query accessing 100,000,](https://wearedevelopers.imgix.net//magazine/articles/803/images/content/g8cGGSLhO1hdQkOHgZxJ-1794237477.png?w=984&auto=compress,format) _As query length increases, the number of possible token combinations rises rapidly, and latency becomes unpredictable under production traffic_ ## Fuzzy matching introduces similar trade-offs. While a more permissive typo tolerance improved usability, it also dramatically increased the number of candidate documents that Elasticsearch needed to evaluate during scoring. Broad synonym expansion had the same effect. Eventually, we realised that search quality features could not simply be stacked indefinitely. Each additional feature effectively consumed part of the latency budget. This changed our approach to search improvements. Rather than enabling every relevance feature at once, we started introducing them gradually and measuring their impact carefully. We also started treating latency regressions as production issues rather than acceptable side effects of ‘better’ search. # The biggest optimisation happened before Elasticsearch A large percentage of marketplace queries contain very little useful information. Users add punctuation, duplicate words or generic commercial phrases like “buy”, “cheap” or “best price”. Those terms rarely improve relevance, but they still increase the amount of work Elasticsearch has to perform during tokenization and scoring. Over time, several preprocessing optimisations produced consistently measurable improvements: - query normalization and lowercasing; - removing punctuation and meaningless tokens; - limiting query length; - stricter typo tolerance settings; - avoiding overly aggressive synonym expansion; - and converting parts of queries into structured filters whenever possible. ![A diagram showing how queries are optimised](https://wearedevelopers.imgix.net//magazine/articles/803/images/content/VLR29qrUJDBcndPBgTsZ-1794237477.png?w=984&auto=compress,format) The last optimisation turned out to be especially important. For example, in many marketplace categories, color behaves more like a structured attribute than free text. A query like “red socks” does not necessarily require Elasticsearch to calculate relevance across every token combination. The system can extract “red” as a color filter before full-text ranking even begins, dramatically reducing the number of candidate documents involved in scoring. ![A diagram showing a huge increase in query length pre-processing does not occur](https://wearedevelopers.imgix.net//magazine/articles/803/images/content/Wvtv0E3k8O0rb2zdMc4t-1794237477.png?w=984&auto=compress,format) _Query preprocessing pipeline: extracting structured filters such as brand, color and SKU before full-text search reduces scoring complexity and keeps Elasticsearch latency predictable under heavy load_ That distinction became critical at scale. Structured filtering is significantly cheaper than full-text relevance calculations. The more context we could extract before Elasticsearch started searching, the more predictable latency became under heavy traffic. Over time, this led to a broader architectural principle: search systems scale better when domain logic removes complexity early instead of forcing the search engine to infer everything dynamically. # Caching solved more problems than additional tuning Another important realization was that marketplace traffic is highly repetitive. Users tend to search for the same brands, categories and popular products, especially during seasonal campaigns or promotions. Certain cities, queries and inventory groups become disproportionately “hot,” creating highly concentrated traffic patterns. Because of that, caching consistently delivered larger performance improvements than endless ranking optimisation. Instead of trying to make every query infinitely efficient, we focused on reducing how often expensive queries needed to reach Elasticsearch at all. Once hot queries and frequently accessed product groups were cached effectively, overall system stability improved substantially during peak traffic periods. This also reduced the pressure created by sudden promotional spikes, where search systems often become unstable long before infrastructure limits are officially reached. # Search quality is usually a business problem One of the biggest misconceptions about search systems is that the quality of rankings is primarily an Elasticsearch problem. In practice, however, search quality depends much more on understanding user intent and marketplace behaviour than on the configuration of the search engine itself. Users search differently depending on what they are trying to accomplish. Some search for exact SKUs or product names. Others search by category, brand, or vague purchase intent. Autocomplete behaviour, product popularity and regional demand patterns also heavily influence what users perceive as ‘good’ search results. For this reason, improving search quality has gradually become less about tuning Elasticsearch and more about understanding the business domain well enough to provide the search engine with better context. This also changed how experimentation worked inside the system. Rather than searching for a single ideal configuration, we began to treat search as an iterative process centred on measurement and controlled rollout. We tested different ranking strategies, typo tolerance settings and query-processing rules gradually through A/B experiments and separate index configurations. In many cases, the best improvements did not come from adding more sophisticated ranking logic. They came from simplifying ambiguous queries, separating different search flows, and reducing unnecessary work inside the relevance pipeline. # Conclusion At scale, Elasticsearch performance problems are often symptoms of architectural decisions made far upstream from the search cluster itself. In our case, the largest improvements did not come from making search more sophisticated. They came from reducing unnecessary complexity: flatter documents, simpler queries, structured filtering, aggressive preprocessing and careful control over expensive relevance features. ## The biggest optimisation was not teaching Elasticsearch to do more work. It was making sure it had less work to do in the first place. ## Related Articles - [SEO in an AI world - Google vs. ChatGPT and survival tips for content creators](https://www.wearedevelopers.com/magazine/534-seo-in-an-ai-world-google-vs-chatgpt-and-survival-tips-for-content-creators) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [A Tour of the PostgreSQL Analytics Ecosystem](https://www.wearedevelopers.com/magazine/792-a-tour-of-the-postgresql-analytics-ecosystem) - [The Web We Broke (And Why AI Agents Are Paying the Price) - AgentCon Berlin](https://www.wearedevelopers.com/magazine/735-the-web-we-broke-and-why-ai-agents-are-paying-the-price-agentcon-berlin) ## Related Videos - [Distributed search under the hood](https://www.wearedevelopers.com/videos/256-distributed-search-under-the-hood) - [Writing a full-text search engine in TypeScript](https://www.wearedevelopers.com/videos/504-writing-a-full-text-search-engine-in-typescript) - [Optimizing Discovery: PostgreSQL's Role in Transforming GetYourGuide's Search](https://www.wearedevelopers.com/videos/1647-optimizing-discovery-postgresql-s-role-in-transforming-getyourguide-s-search) - [Scaling: from 0 to 20 million users](https://www.wearedevelopers.com/videos/676-scaling-from-0-to-20-million-users) ## Related Jobs - [Software Development Engineer 5](https://www.wearedevelopers.com/jobs/48599-software-development-engineer-5) at **WEX** - [Senior Software Engineer - Semantic Data Lake (Open)SDE3](https://www.wearedevelopers.com/jobs/48597-senior-software-engineer-semantic-data-lake-open-sde3) at **WEX** - [Software Development Engineer 1](https://www.wearedevelopers.com/jobs/48592-software-development-engineer-1) at **WEX** - [Senior Specialist, Product Operations](https://www.wearedevelopers.com/jobs/48622-senior-specialist-product-operations) at **Delivery Hero** - [Director of Engineering - Data Platform SDM5](https://www.wearedevelopers.com/jobs/48598-director-of-engineering-data-platform-sdm5) at **WEX** - [Software Development Engineer 1](https://www.wearedevelopers.com/jobs/48593-software-development-engineer-1) at **WEX**