Senior Software Engineer (AI Infra + Systems)

This position focuses on improving the performance, reliability, and efficiency of AI-powered applications while expanding the organization’s model-serving capabilities. You’ll work across backend engineering, distributed systems, and production AI infrastructure, with an emphasis on optimizing inference performance, reducing latency, and evaluating opportunities to move from managed AI services toward self-hosted models.

What You’ll Do

• Build & Maintain AI Backend Services: Develop and optimize production microservices using Python and Scala, supporting AI integrations, application APIs, and distributed backend systems.
• Improve Scalability & Performance: Identify and eliminate bottlenecks involving latency, concurrency, throughput, and resource utilization across high-traffic applications.
• Optimize Model Inference: Evaluate and implement model-serving architectures, including self-hosted inference, GPU utilization, autoscaling, and cost optimization.
• Own Production Reliability: Manage the performance, monitoring, deployment, and operational health of AI services running in Kubernetes.
• Drive Technical Improvements: Collaborate across engineering teams to improve system architecture, measure performance, and deliver scalable AI capabilities.

• 5 + years of experience minimally. Fluent on the backend with Python and Scala
• Experience designing, developing, and maintaining distributed systems and microservices in production.
• Hands-on experience with Kubernetes, containerized applications, and cloud infrastructure.
• Demonstrated ability to diagnose and resolve performance bottlenecks involving latency, concurrency, and system throughput.
• Experience deploying or operating ML models through production APIs, with an understanding of inference performance.
• Practical knowledge of MLOps principles related to model serving, monitoring, deployment, and operational reliability.
• Familiarity with LLM-powered applications and the backend architecture supporting generative AI systems.
• Strong understanding of production observability, including logging, metrics, tracing, and performance monitoring.

This Jobot Job is hosted by: Grant Greenhalgh Are you a fit? Easy Apply now by clicking the “Apply Now” button and sending us your resume. Salary: $170,000 - $200,000 per year

A bit about us:

We’re an early-stage AI startup on a mission to redefine how users interact with intelligent systems. We’re building something brand new and we’re looking for talent to help us scale our platform services.

Why join us?

• Fully Remote
• Well-funded
• Senior team of proven engineers
• Autonomy and upward mobility
• Competitive compensation
• Lucrative Equity
• Great healthcare benefits