AI engineer

Hiring Remotely in USA Entry level Remote Hiring Remotely in USA Entry level Build AI agent evaluation and trace intelligence features, run benchmarks and experiments on Mastra primitives, and use findings to improve agent accuracy, configuration, and behavior. Develop tools that help developers understand agent performance and enable automatic self-improvement through evaluations, suggestions, and rollouts. This hands-on role combines experimentation, benchmarking, and production framework engineering in a fully remote environment. The summary above was generated by AI

We’re looking for an AI engineer. You’ll work on features that help developers understand their agents’ behavior, evaluate their performance, and improve them over time.

This is a hands-on engineering role. You’ll move between experiments, benchmarks, and product code, taking what you learn from evaluating agents and using it to shape the framework.

Why:

• We’re starting to build more and more agentic features into Mastra: Trace intelligence, and we’re working on various Agent Learning features to automatically self-improve agents (Autoevals + Suggestions, Rollouts).
• In addition, we’ve been increasingly running Mastra primitives like memory on industry-standard benchmarks and using the results to fine-tune configuration.

What you’ll do

Build the features that make this possible: tools for understanding traces, evaluating agent behavior, and turning results into improvements.

Run experiments on Mastra’s primitives, investigate where they perform well and where they fall short, and use those findings to improve their configuration and behavior. You’ll help turn that work into capabilities developers can use in their own applications.

You have:

• Shipped agents and agentic applications into production
• Tuned and tweaked them to improve accuracy
• You need to be able to benchmark your results against existing datasets, or intelligently creating.

About Mastra

Mastra is the open-source TypeScript framework for building AI agents. Built by the team behind Gatsby, backed by Y Combinator and Spark Capital, with $35M raised total. We went from 0 to 27k+ stars and 1M+ weekly npm downloads in under 18 months.

Small team, shipping fast, fully remote. Most of us work across North American and European time zones., Designs and ships production multi-agent AI systems for KYB, underwriting, case review, and risk monitoring. Owns agent architecture, retrieval, tool integrations, evaluations, observability, MLOps, reliability, and compliance guardrails. Requires strong Python engineering, agent framework expertise, distributed systems experience, Kubernetes operations, and production LLM deployment skills. Mentors engineers and partners with product, security, compliance, and applied science teams. Remote employees travel to Orlando at least twice annually. Top Skills: Anthropic ClaudeArgocdAWSBraintrustClaude Agent SdkDatadogEksKafkaKubernetesLangchainLangfuseLanggraphLangsmithLlamaMcpMistralNode.jsOpenaiOpenai SdkOpensearchPgvectorPostgresPythonRedisRedshiftTerraformTypescriptVector Databases, Samsara

Designs, develops, tests, validates, and deploys AI/ML models across generative AI, classification, prediction, computer vision, NLP, and reinforcement learning. Builds data and model-training pipelines, leverages cloud platforms, evaluates model performance, documents methodologies, collaborates with technical teams, and guides prototype-to-production transitions. Requires five years of relevant experience and an active TS/SCI clearance with polygraph.

Remote or Hybrid Easy Apply 137K-231K Annually Senior level 137K-231K Annually Senior level Artificial Intelligence * Cloud * Computer Vision * Hardware * Internet of Things * Software Architect and own Samsara’s marketing data and AI foundation, including Databricks pipelines, data models, semantic layers, and self-service analytics infrastructure. Lead cross-functional initiatives, ensure data quality and reliability, build and evaluate production LLM and agent systems, automate workflows with AI, and mentor engineers and analysts. Partner with Marketing, Sales, BI, and Data Science teams to turn ambiguous business needs into scalable data products. Top Skills: BigQueryClaude CodeCursorDatabricksDbtLlmsPythonSalesforceSnowflakeSQLStreamlit

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

• Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
• Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
• Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
• Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
• Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
• Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute