> Markdown version of [/jobs/ext/3637432-ai-engineer](https://www.wearedevelopers.com/jobs/ext/3637432-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI engineer

- **Company:** Kepler Software, Inc.
- **Location:** United States (Remote available)
- **Experience:** Expert
- **Salary:** $160,000.0 - $250,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Business Analytics Applications, Data Analysis, Computer Vision, Cloud Computing, Cyber Security, Databases, Distributed Systems, Python (Programming Language), Machine Learning, Marketing Information Systems, Natural Language Processing, Open Source Technology, TypeScript, Reinforcement Learning, Large Language Models, Apache Spark, Agentic-AI, Data Layers, Gatsby, Kubernetes, Information Technology, Machine Learning Operations, Databricks
- **Published:** October 8, 2026
- **Apply:** https://www.builtincolorado.com/job/ai-engineer/11549368?handler=ApplyRedirect

## About the Role

Designs, develops, tests, validates, and deploys AI/ML models across generative AI, classification, prediction, computer vision, NLP, and reinforcement learning. Builds data and model-training pipelines, leverages cloud platforms, evaluates model performance, documents methodologies, collaborates with technical teams, and guides prototype-to-production transitions. Requires five years of relevant experience and an active TS/SCI clearance with polygraph.

## Description

Hiring Remotely in USA
Entry level
Remote
Hiring Remotely in USA
Entry level
Build AI agent evaluation and trace intelligence features, run benchmarks and experiments on Mastra primitives, and use findings to improve agent accuracy, configuration, and behavior. Develop tools that help developers understand agent performance and enable automatic self-improvement through evaluations, suggestions, and rollouts. This hands-on role combines experimentation, benchmarking, and production framework engineering in a fully remote environment.
The summary above was generated by AI

We're looking for an AI engineer. You'll work on features that help developers understand their agents' behavior, evaluate their performance, and improve them over time.

This is a hands-on engineering role. You'll move between experiments, benchmarks, and product code, taking what you learn from evaluating agents and using it to shape the framework.

Why:

 * We're starting to build more and more agentic features into Mastra: Trace intelligence, and we're working on various Agent Learning features to automatically self-improve agents (Autoevals + Suggestions, Rollouts).
 * In addition, we've been increasingly running Mastra primitives like memory on industry-standard benchmarks and using the results to fine-tune configuration.

What you'll do

Build the features that make this possible: tools for understanding traces, evaluating agent behavior, and turning results into improvements.

Run experiments on Mastra's primitives, investigate where they perform well and where they fall short, and use those findings to improve their configuration and behavior. You'll help turn that work into capabilities developers can use in their own applications.

You have:

 * Shipped agents and agentic applications into production
 * Tuned and tweaked them to improve accuracy
 * You need to be able to benchmark your results against existing datasets, or intelligently creating.

About Mastra

Mastra is the open-source TypeScript framework for building AI agents. Built by the team behind Gatsby, backed by Y Combinator and Spark Capital, with $35M raised total. We went from 0 to 27k+ stars and 1M+ weekly npm downloads in under 18 months.

Small team, shipping fast, fully remote. Most of us work across North American and European time zones., Designs and ships production multi-agent AI systems for KYB, underwriting, case review, and risk monitoring. Owns agent architecture, retrieval, tool integrations, evaluations, observability, MLOps, reliability, and compliance guardrails. Requires strong Python engineering, agent framework expertise, distributed systems experience, Kubernetes operations, and production LLM deployment skills. Mentors engineers and partners with product, security, compliance, and applied science teams. Remote employees travel to Orlando at least twice annually.
Top Skills: Anthropic ClaudeArgocdAWSBraintrustClaude Agent SdkDatadogEksKafkaKubernetesLangchainLangfuseLanggraphLangsmithLlamaMcpMistralNode.jsOpenaiOpenai SdkOpensearchPgvectorPostgresPythonRedisRedshiftTerraformTypescriptVector Databases, Samsara