> Markdown version of [/jobs/ext/3962480-senior-ai-engineer](https://www.wearedevelopers.com/jobs/ext/3962480-senior-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior AI Engineer

- **Company:** Procore
- **Location:** Austin, TX, United States
- **Experience:** Expert
- **Salary:** $140,960.0 - $193,820.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Business Analytics Applications, Extract Transform Load (ETL), Data Warehousing, Python (Programming Language), Machine Learning, Project Management Software, Data Streaming, TypeScript, Large Language Models, Snowflake, Langfuse — LLM Observability and Analytics Platform, Agentic-AI, Information Technology, Data Pipelines, Databricks
- **Published:** October 27, 2026
- **Apply:** https://www.thejobnetwork.com/job/c9d1f94c-b945-455a-b393-3981355f73ca/senior-ai-engineer

## About the Role

* 5+ years of experience in software, AI, or ML engineering.
 * Strong proficiency in Python; experience with TypeScript is a plus.
 * Experience evaluating LLM systems, such as LLM-as-judge, rubric design, goldset curation, or measuring judge agreement.
 * Experience with LLM observability or evaluation tools such as Langfuse, Braintrust, or similar.
 * Experience building data pipelines, ETL processes, or streaming data systems.
 * Experience with analytics platforms and data warehouses such as Databricks or Snowflake.
 * Ability to communicate eval results clearly and influence the teams that build on them.
 * Degree in computer science or a related technical field, or equivalent practical experience.

## Description

We're looking for a Senior AI Engineer to join Procore's Evals & Goldsets team, which measures and improves the quality of the AI agents we ship to the construction industry. You'll build and run the systems that tell our AI teams how well their agents are doing: production tracing in Langfuse and Databricks, our online LLM judge, and the goldsets and benchmarks behind our offline evals.

As a Senior AI Engineer, you will partner with data and ML engineers to make our evals faster, cheaper, more trustworthy, and harder to saturate, and you'll turn production traces into feedback loops that improve agent behavior. We're a small, fast-moving team that pairs hands-on building with architecture and review, and we value engineers who are rigorous about measurement and comfortable with ambiguity. If you want to shape how AI in construction gets tested and improved, come join us.

This hybrid position reports to the Engineering Manager for Evals & Goldsets and will be based in our Austin, TX office.

What you'll do:

 * Improve our online evals: LLM judge accuracy, calibration, and the infrastructure that runs it in production.
 * Build and maintain production trace infrastructure with Langfuse and Databricks.
 * Build and maintain data pipelines for the goldsets behind our evals, including annotation and quality checks.
 * Build agentic loops that mine production traces for failures and turn them into new eval cases and agent improvements.
 * Improve the runtime, cost, and reliability of our online and offline eval pipelines.
 * Contribute code and architecture, review designs, and mentor other engineers on the team.