> Markdown version of [/jobs/ext/4617977-lead-data-platform-and-ml-manager](https://www.wearedevelopers.com/jobs/ext/4617977-lead-data-platform-and-ml-manager). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Platform and ML Manager

- **Company:** LIGHTHOUSE
- **Location:** United, WV, United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** OpenTofu, Artificial Intelligence, Amazon Web Services, Automation of Tests, Continuous Integration, Information Engineering, Data Infrastructure, Github, Python (Programming Language), Operational Databases, Standard Sql, GitHub Copilot, Claude Code, Apache Spark, Data Lakes, Pyspark, Information Technology, Terraform, Databricks
- **Published:** November 7, 2026
- **Apply:** https://www.thejobnetwork.com/job/63fbea50-009e-44c8-b6de-21602dd05c20/lead-data-platform-and-ml-manager

## About the Role

· Data engineering. 5 or more years building production data pipelines. Strong SQL and Python, production experience with Spark or PySpark, and dbt for transformation and testing.

\n

· Databricks. Hands-on production experience with Unity Catalog, Delta Lake, Lakeflow, Databricks SQL, and Databricks AI functions.

\n

· AWS and infrastructure as code. Running Databricks on AWS, with infrastructure as code in Terraform or OpenTofu and workloads as Databricks Asset Bundles.

\n

· CI/CD. Pipeline ownership using GitHub Actions, with automated testing and controlled promotion across environments., · Honest about uncertainty. You will say when a model is not good enough yet, including when that is inconvenient.

\n

· Writer and communicator. We are remote and asynchronous, and much of this role is documentation that a regulator or auditor will read. Excellent written and verbal communication skills are required., · Bachelor's degree in Computer Science, Statistics, Mathematics, Engineering, or a related quantitative field, or equivalent practical experience. A master's degree is helpful. Insurance, actuarial, healthcare, or credit risk modeling experience is a plus.

## Description

· AI-assisted development. We build with AI every day, and we expect our data and AI engineer to do the same. You use tools like Claude Code, Databricks Assistant, or GitHub Copilot as part of your normal workflow, and you review their output as carefully as a teammate's., · Business-curious. You will want to understand how the business works, not only how the pipeline runs. You work closely with our Lead Developer and with business, analytics, and compliance teams.