> Markdown version of [/jobs/ext/1183015-senior-data-platform-data-product-engineering-lead](https://www.wearedevelopers.com/jobs/ext/1183015-senior-data-platform-data-product-engineering-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Platform / Data Product Engineering Lead

- **Company:** Tata Consultancy Services Limited
- **Location:** Irvine, CA, United States
- **Experience:** Expert
- **Salary:** $100,000.0 - $120,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Microsoft Azure, Software Documentation, Continuous Integration, Directed Acyclic Graph (Directed Graphs), Data Discovery, Information Engineering, Data Governance, Data Infrastructure, Data Masking, Data Sharing, Github, Python (Programming Language), Metadata Standards, SQL Databases, Systems Integration, Management of Software Versions, Enterprise Data Management, Datadog, Grafana, Data Lakes, Data Lineage, Api Design, Marketplace, Databricks
- **Published:** July 4, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=01f3efb92ad41f0c

## About the Role

* 10+ years of experience in Data Engineering / Data Platform roles

 * Strong hands-on expertise in:

o Databricks (Delta Lake, workflows, DAG)
o Apache Airflow / Astronomer
o Python, SQL, DBT ,AWS

 * Proven experience implementing CI/CD frameworks (Harness, GitHub Actions, Azure DevOps)

 * Deep understanding of:

o Data governance (catalogs, lineage, contracts, metadata)
o Data quality and masking techniques
o Enterprise data platforms and marketplace ecosystems

 * Experience with API-based integrations (e.g., entitlement systems like AccessCentral)

 * Monitoring/observability tools (e.g., Datadog)

## Description

We are looking for a Senior Data Platform / Data Product Engineering Lead to drive enterprise-scale data product lifecycle enablement across modern data platforms. This role will lead the design, standardization, and adoption of a paved path for data creators, enabling self-service, governed, and scalable data product development.
The role requires deep expertise in Databricks, Airflow (Astronomer), CI/CD automation, data governance, and data marketplace constructs, along with the ability to lead platform transformation initiatives and mentor engineering teams., We are looking for a Senior Data Platform / Data Product Engineering Lead to drive enterprise-scale data product lifecycle enablement across modern data platforms. This role will lead the design, standardization, and adoption of a paved path for data creators, enabling self-service, governed, and scalable data product development.
The role requires deep expertise in Databricks, Airflow (Astronomer), CI/CD automation, data governance, and data marketplace constructs, along with the ability to lead platform transformation initiatives and mentor engineering teams.
________________________________________
Core Responsibilities

 1. Platform Strategy & Self-Service Enablement
 * Define and implement a self-service data platform strategy to reduce onboarding friction.

 * Lead automated provisioning of:

o Databricks workspaces (via DevHub)
o Airflow/Astronomer environments
o Access and entitlements (AccessCentral APIs)

 * Establish isolated, stable development environments for federated teams.

 * Drive platform observability by integrating metrics into tools lik e Datadog.

________________________________________

 2. Data Discovery, Access & Governance
 * Architect and implement enterprise-wide data discovery and marketplace enablement.

 * Drive adoption of:

o Data contracts
o Metadata standards
o Domain-aligned catalogs (Unity Catalog)

 * Enable secure access to curated, masked datasets in dev and production environments.

 * Implement tagging, access patterns, and entitlement automation.

 * Partner with risk/compliance teams to enforce regulatory and governance controls (BFSI-aligned)., o Airflow DAG libraries
o DBT-based transformation models

 * Ensure:

o Data quality and consistency
o Embedded governance and compliance policies

 * Enable concurrent development using standardized patterns and environments.

________________________________________

 4. CI/CD, Automation & Deployment
 * Define and enforce standard CI/CD pipelines across data products:

o Harness (or equivalent)
o Databricks Asset Bundles (DAB)

 * Automate:

o DAG deployments (Airflow/Astronomer)
o DBT pipeline releases

 * Reduce manual interventions and ensure consistent, repeatable deployments.

 * Improve release reliability with feedback loops, notifications, and monitoring.

________________________________________

 5. Data Product Publishing & Marketplace Enablement
 * Drive publishing of data products to:

o Unity Catalog
o Enterprise Data Marketplace

 * Define and enforce:

o Documentation standards
o Data ownership models
o Versioning and contract management

 * Enable cross-domain data sharing with embedded governance and access controls.

________________________________________

 6. Operations, Observability & Reliability
 * Establish a scalable operating model for data product support.

 * Implement:

o Monitoring dashboards (Datadog)
o Data quality frameworks
o Usage and performance metrics tracking

 * Improve visibility into:

o Pipeline health
o Data lineage
o Access and consumption patterns

 * Lead incident management, root cause analysis, and escalation processes.

________________________________________

 7. Transformation, Roadmap & Innovation
 * Drive execution of platform priorities such as:

o Data contract activation strategy
o Domain catalog integration
o Data masking in development environments
o Data quality frameworks
o DBT adoption and POCs

 * Lead maturity uplift from:

o Manual, fragmented workflows standardized, automated paved paths

 * Champion continuous improvement and innovation in developer experience.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [5 steps for running a Kubernetes environment at scale](https://www.wearedevelopers.com/videos/88-5-steps-for-running-a-kubernetes-environment-at-scale)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [All your telemetry data from any source in one place](https://www.wearedevelopers.com/videos/57-all-your-telemetry-data-from-any-source-in-one-place)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)

## Related Articles

- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)