> Markdown version of [/jobs/ext/4258345-ai-engineer](https://www.wearedevelopers.com/jobs/ext/4258345-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Engineer - **Company:** RAVENNA L.L.C. - **Location:** United States (Remote available) - **Experience:** Expert - **Salary:** $160,000 - $200,000 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Data Analysis, Automation of Tests, Big Data, Code Review, Information Retrieval, Knowledge-Based Systems, Machine Learning, Language Modeling, Cloud Services, Search Technologies, Systems Architecture, Datadog, Evaluation Pipelines, Retrieval-Augmented Generation, Large Language Models, Model Validation, Indexer, Backend, Agentic-AI, Build Management, AI Platforms, Attention Mechanisms, Information Technology, Production Code, Machine Learning Operations - **Published:** October 31, 2026 - **Apply:** https://www.builtincolorado.com/job/ai-engineer/9223594?handler=ApplyRedirect ## About the Role You are a strong software engineer first. You have at least five years of experience building production systems and are comfortable designing complex systems that are reliable, maintainable, and scalable. You think carefully about architecture, testing, observability, performance, and operational reliability. LLM systems experience You have built systems around large language models in production environments. You understand that successful AI products require thoughtful system design, not just calling an API. You have experience designing prompts, managing context, orchestrating tool usage, and improving the reliability of model outputs. Evaluation mindset You care deeply about measuring quality. You have experience building evaluation frameworks, designing datasets, and using experiments to guide improvements to prompts, models, and system architecture. Retrieval and embedding systems You have experience with retrieval augmented generation systems and understand how retrieval quality affects model performance. You are comfortable working with embeddings, vector databases, and semantic search, and you understand how these approaches differ from traditional search and indexing systems. LLM fundamentals You understand how modern language models work at a conceptual level and can speak about transformers, attention mechanisms, and the tradeoffs between different model architectures. AI system intuition You understand the strengths and weaknesses of LLMs in practice. You think about hallucinations, prompt sensitivity, context limits, latency, and cost when designing systems. Experimentation and iteration You are comfortable running experiments across prompts, models, and system designs. You use evaluation data and real world feedback to guide improvements. ## Description The AI Engineer will design and build production systems, focusing on LLM integration, system reliability, evaluation frameworks, and high-quality product features while maintaining strong engineering practices. The summary above was generated by AI, At Ravenna, we are looking for experienced engineers with a passion for AI and a strong track record of building and shipping production systems. Our team is combining the latest advances in large language models with proven machine learning techniques to build modern, intelligent product experiences. This role requires strong engineering fundamentals and a deep curiosity about how AI systems work in practice. You will design, build, and operate LLM powered systems that are reliable, scalable, and deeply integrated into real product workflows. You will work across the stack. This includes designing AI systems, building backend infrastructure, implementing product features, and iterating quickly based on evaluation and real user feedback. We care deeply about strong engineering practices and expect our AI engineers to think carefully about system design, observability, performance, and reliability. If you are excited about applying cutting edge AI to disrupt large and established markets, we want to talk to you. ResponsibilitiesBuild production AI systems Design and implement systems that integrate LLMs into real product workflows. This includes prompt pipelines, tool integration, structured outputs, and systems that combine models with traditional software components. Design reliable LLM architectures Develop robust systems that account for model limitations and failure modes. Build safeguards, retries, evaluation loops, and observability into LLM powered features so they behave reliably in production. Develop evaluation and experimentation frameworks Create evaluation datasets and tooling that allow the team to measure model performance, iterate on system design, and prevent regressions as models and prompts evolve. Build retrieval and knowledge systems Design and optimize retrieval pipelines that power LLM applications. This includes chunking strategies, indexing approaches, vector search, and improving the quality and speed of knowledge retrieval. Ship high quality product features Collaborate with product, design, and engineering teammates to build polished features that leverage AI in ways that feel natural and valuable to users. Maintain strong engineering standards Write clean, maintainable, well tested code. Contribute to system architecture, code reviews, and engineering processes that ensure the platform remains reliable and scalable., Remote United States Easy Apply 173K-255K Annually Senior level 173K-255K Annually Senior level Big Data * Fintech * Mobile * Payments * Financial Services Build and operate Affirm's AI Platform, including LLM and MCP gateways, agent runtimes, evaluation services, and observability systems. Design secure, highly available backend infrastructure with identity, quota, routing, caching, failover, cost attribution, and governance controls. Own technical projects from design through launch and on-call, establish engineering standards, collaborate across security and infrastructure teams, and mentor engineers while supporting production AI workloads at scale. Top Skills: AWSAws BedrockChatgptClaudeKotlinKubernetesLitellmMcpMySQLPython Optum Machine Learning Engineer Yesterday In-Office or Remote 99K-176K Annually Mid level 99K-176K Annually Mid level Artificial Intelligence * Big Data * Healthtech * Information Technology * Machine Learning * Software * Analytics Build and deploy production-grade AI/ML software, APIs, and applications for healthcare provider use cases. Develop prompt-based AI capabilities, integrate enterprise data and cloud services, create automated tests and evaluation pipelines, and ensure security, observability, scalability, and reliability. Collaborate with product, architecture, data science, and platform teams while owning technical work from proof of concept through production. Top Skills: APIsCi/CdCloud PlatformsDockerGitJavaJava EeJwtModel EvaluationMongoDBMySQLOauthOpenshiftPrompt EngineeringReactRestSplunkSpring Boot Circle (circle.so) Lead Engineer, AI Platform Yesterday In-Office or Remote 2 Locations Senior level Senior level Artificial Intelligence * Consumer Web * Digital Media * Information Technology * Social Impact * Software Build and own evaluation infrastructure, observability tooling, datasets, annotation workflows, and diagnostic systems for production AI agents. Lead experiments across prompts, models, routing, caching, cost, and latency. Diagnose failures across planning, tool use, and agent execution, while partnering with AI Core engineering. Manage and grow the AI Quality team as a hands-on player-coach, contributing directly to production code and technical direction. Top Skills: Ai AgentsAnnotation WorkflowsBraintrustCachingCi/CdEvaluation FrameworksLangsmithLlmsModel RoutingObservability ToolingPythonRuby On Rails What you need to know about the Colorado Tech Scene With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation. Key Facts About Colorado Tech * Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey) * Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3 * Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech * Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook) * Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures * Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute