> Markdown version of [/jobs/ext/4313130-staff-software-engineer-ai-gateway](https://www.wearedevelopers.com/jobs/ext/4313130-staff-software-engineer-ai-gateway). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Staff Software Engineer, AI Gateway

- **Company:** Docker
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Salary:** $170,350.0 - $275,550.0
- **Contract:** Permanent contract
- **Skills:** Abstraction Layers, Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Distributed Systems, Network Control, OAuth, OpenAI, Data Logging, Network Routers, Load Balancing, Cloud Platform System, Retrieval-Augmented Generation, Large Language Models, Caching, Agentic-AI, Rate Limiting, Kubernetes, Information Technology, CrewAI, Virtual Agents, Api Design, Api Gateway, Docker, Microservices
- **Published:** October 18, 2026
- **Apply:** https://www.dice.com/job-detail/a9c7acf5-fad5-4abc-805e-9880b4f23419

## About the Role

* 8+ years of backend engineering experience, with a track record of owning production-grade systems at significant scale
 * Deep specialization in distributed systems, cloud-native infrastructure, and API design - recognized by peers and cross-functional partners as a go-to expert, not just experienced
 * Experience building secure, high-throughput services (gateways, proxies, load balancers, policy engines), and a history of making judgment calls other engineers rely on
 * Fluency in Go and/or Rust
 * A track record of identifying systemic gaps - in architecture, process, or tooling-- and driving the fix across team boundaries, not just within your own
 * A strong product mindset, with real experience shaping technical direction rather than only executing against it
 * Comfortable operating with wide latitude: given a problem and significant autonomy in how to solve it, including when and who to pull in
 * Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience

Bonus Qualifications

 * Prior experience with OpenAI, Anthropic, or similar LLM APIs
 * Familiarity with RAG architectures, vector databases, or agent frameworks (e.g., LangChain, AutoGPT, CrewAI)
 * Experience with policy engines (e.g., OPA), observability frameworks (e.g., OpenTelemetry), or API gateways
 * Understanding of OAuth 2.1, secret management, and cloud IAM systems
 * Experience with Kubernetes, Docker, and microservices architecture
 * A track record of setting technical standards that other teams or engineers went on to adopt

## Description

* Set technical direction for core systems powering the AI Gateway 0 the model router, MCP gateway, and control plane - and make sure architectural decisions hold up as usage scales by 10x
 * Consistently identify gaps in our infrastructure, process, or tooling before they become blockers, and drive the fix from idea to production
 * Take a significant, hands-on role in defining, delivering, and implementing the Gateway's technical strategy, in close partnership with engineering leadership
 * Own the hardest, highest-ambiguity problems in the system: dynamic model selection, auto-failover, cost-based routing, and policy enforcement that has to work end-to-end
 * Act as the go-to expert for secure credential storage, session summarization, caching, and rate limiting - the capabilities everything else depends on
 * Build the infrastructure that lets us, and our customers, evaluate, replay, and backtest agent and LLM behavior with confidence
 * Mentor senior and mid-level engineers, and raise the technical bar for the team through code, design review, and example
 * Partner with product, design, and leadership to shape not just what the control plane UI does, but where the product goes next

Key Problems You'll Help Solve

 * Define the unified abstraction layer across diverse model and tool providers (OpenAI, Anthropic, Google, AWS Bedrock) that the rest of the org builds on
 * Architect secure, scalable identity and credential vaulting for tool and model access, trusted enough to run at enterprise scale
 * Build the infrastructure for real-time and historical analytics of AI agent behavior that leadership and customers both rely on
 * Guarantee that policy enforcement and logging work end-to-end - from prompt to tool to response - with no gaps an auditor (or attacker) could find
 * Set the bar for developer experience through intuitive APIs and first-class observability