Scaling Distributed Queues for AI workloads

Almost every asynchronous action at Meta passes through a single system most people have never heard of. The Facebook Ordered Queueing Service (FOQS), a fully managed, horizontally scalable priority queue, moves close to a trillion items per day for 300+ engineering teams, and it has become critical infrastructure for AI at Meta's scale. Async LLM inference, Llama serving, GenAI image generation, and AI compute demand control all ride on it. This talk traces how a queue originally built to absorb massive backlogs and prioritize work across highly heterogeneous producers and consumers grew into the reliability layer beneath Meta's AI stack supporting critical use cases like Ads, Integrity, AI Training and Whatsapp message delivery. It explains how one system came to serve hundreds of teams without them stepping on each other, how it keeps the most important work moving under enormous load, and how it evolved from isolated regional deployments into a globally distributed service that delivers region-level disaster recovery in seconds with zero client-visible downtime. The finale looks at how that same queue now acts as a control plane for shaping AI compute demand.