Data Engineer & Platform Manager
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+8 more
Job description
Data Engineering & Platform Manager - Azure Databricks and Medallion Architecture Impact Business Group is recruiting a Data Engineering & Platform Manager for a manufacturing client. Youβll take hands-on ownership of its enterprise Azure Databricks platform. Your first assignment is to assess the legacy environment and the newer implementation, understand how teams use both, and lead the work to make the newer platform meet the businessβs reporting, analytics, and AI needs. Youβll report to the VP of Data, Analytics & AI and work together to set the technical roadmap. This is an individual contributor role with no direct reports. Youβll make architecture decisions and build, test, and implement changes yourself. What youβll do Trace critical data flows from source systems to downstream reports. Compare the two environments, decide what to keep or rework, and validate changes before moving business-critical workloads. Define and implement the medallion architecture. Set clear standards for bronze, silver, and gold layers, and build and maintain production pipelines from ERP, CRM, and other enterprise systems. Own Databricks workspace administration, compute configuration, and pipeline operations. Monitor reliability, performance, and spend, and resolve production issues. Strengthen security and governance with Unity Catalog. Implement access controls, track data lineage and ownership, and establish data quality checks and monitoring. Work with Finance, Supply Chain, and analytics teams to reconcile data and define what must pass before a workload moves. Document pipeline standards and operational runbooks
Requirements
Microsoft Windows Azure, Artificial Intelligence (AI), Manufacturing Management, Technical Recruiting, Testing, Data Analysis, Customer Relationship Management (CRM), ERP (Enterprise Resource Planning), Systems Administration/Management, Problem Solving Skills, 7-12 years of data engineering experience, including 3+ years of hands-on enterprise Databricks administration and pipeline development. Production experience with Azure Databricks, Delta Lake, Unity Catalog, medallion architecture, and pipeline orchestration using Databricks Workflows, Azure Data Factory, or equivalent tools. Strong SQL and Python skills, proficiency with Azure data services, and experience integrating ERP data into an enterprise platform. Experience implementing data access controls, lineage, and data quality standards, and translating business requirements into architecture and implementation decisions. Bachelorβs degree in computer science, data engineering, information systems, or a related field. Preferred experience PySpark, SAP, Microsoft Dynamics GP, Databricks compute and cost optimization, or data engineering in regulated manufacturing. Travel up to 15% as needed.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦