> Markdown version of [/jobs/ext/3808039-ai-engineer](https://www.wearedevelopers.com/jobs/ext/3808039-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI Engineer

- **Company:** The Vertafore Way
- **Location:** Denver, CO, United States (Remote available)
- **Experience:** Expert
- **Salary:** $175,000.0 - $210,000.0
- **Contract:** Permanent contract
- **Skills:** LangGraph Framework, Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Systems Engineering, Information Extraction, Python (Programming Language), Machine Learning, Automation of Marketing, Performance Tuning, Regression Testing, Software Engineering, Software Systems, Retrieval-Augmented Generation, Large Language Models, Multi-Agent Systems, Model Validation, Llamaindex, Backend, Langfuse — LLM Observability and Analytics Platform, Fastapi, Build Management, LangSmith, Kubernetes, Information Technology, Low Latency, Production Code, Free and Open-Source Software, Restful APIs, Model Context Protocol, Human in the Loop, Docker
- **Published:** October 17, 2026
- **Apply:** https://www.builtincolorado.com/job/sr-ai-engineer/11721707?handler=ApplyRedirect

## About the Role

· Expert-level Python, including writing idiomatic, typed, tested, production-quality code and using async programming

· Solid understanding of ML fundamentals, including model training and generalization, overfitting, bias and variance, embeddings, classification and ranking metrics, and statistical significance

· Deep understanding of how large language models behave in practice, including failure modes, non-determinism, context limits, and strategies for reliable outputs

· Rigorous, experimental mindset; you form hypotheses, measure, and let data drive decisions rather than intuition about model output, · Bachelor's degree in Computer Science, Engineering, Physics, Mathematics, or a related field, or equivalent practical experience

· 7+ years of professional software engineering experience, with significant time building backend services and systems in Python

· 2+ years building and shipping LLM-powered applications or agentic systems to production

· Hands-on production experience with LangGraph (or comparable agent orchestration frameworks such as LangChain, LlamaIndex, or custom orchestration)

· 5+ years designing and building RESTful APIs (experience with FastAPI or similar frameworks preferred)

· Demonstrated experience designing and running evaluations for ML or LLM systems, and using results to drive measurable quality improvements

· Experience with cloud platforms (AWS preferred), Docker, and CI/CD pipelines

Preferred Qualifications:

· Experience with LLM observability and evaluation tooling (such as LangSmith, Langfuse, Braintrust, or similar)

· Experience with fine-tuning, embedding model selection, or classical ML pipelines

· Experience in insurance, financial services, healthcare, or another regulated industry

· Experience with Model Context Protocol (MCP) or building tool integrations for agents

· Open-source contributions or public writing on AI engineering

Additional Requirements and Details:

 * Travel required up to 10% of the time.

 * Located and working from an office location.

 * Occasional lifting and/or moving up to 10 pounds., The selected candidate must be legally authorized to work in the United States.

## Description

As a Senior AI Engineer, you will design, build, and operate the AI systems behind Vertafore's next generation of products for agencies, MGAs, and carriers. You will build production-grade agents that can reason through complex, multi-step insurance workflows, and you will be responsible for making those systems reliable, measurable, and safe in a highly regulated industry.

You will work in small pods with complete agency to build end to end products. This role is hands-on. You'll be expected to directly contribute to the successful implementation and launch of critical features and empowered to make the decisions needed to get there. Vertafore designs these pods as end to end development teams so fluidity going up and down the stack is expected.

We are looking for an engineer who is strong in software and systems engineering, understands machine learning well, and has a habit of shipping. You use the latest agentic coding tools to work faster, and you hold yourself fully accountable for the quality, security, and correctness of what you ship.

Core Requirements and Responsibilities:

Essential job functions included but are not limited to the following:

· Design, build, and deploy complex reasoning agents and multi-agent systems for production workloads using LangGraph and related Python orchestration frameworks

· Architect agent workflows, including state management, tool use, memory, planning, human-in-the-loop checkpoints, error recovery, and fallback behavior

· Design and build clean, well-documented, versioned APIs that expose AI capabilities to Vertafore products and to external partners

· Define and own evaluation strategy for AI systems. This includes building eval datasets, writing graders (programmatic, model-based, and human review), running offline and online evaluations, and turning results into concrete improvements

· Set up regression testing for prompts, models, and agent behavior so that changes ship with measurable confidence

· Instrument AI systems for observability, including tracing, latency, cost, token usage, failure modes, and quality metrics in production

· Build retrieval and context pipelines (RAG, document processing, structured extraction) over insurance data such as policies, submissions, and compliance documents

· Troubleshoot and resolve production issues across the AI stack, from model behavior to infrastructure