Data Platform Engineer (Infrastructure & Data Engineering)

Design and maintain CI/CD pipelines: Develop and manage continuous integration and deployment workflows using DevOps tools (Azure DevOps, GitHub Actions) for both infrastructure and application deployments.

• Infrastructure as Code (IaC): Write and maintain Terraform code to provision and manage Azure infrastructure and Azure Databricks resources (Unity Catalog, workspaces, networking, compute policies).
• Application development and automation: Create and support platform tooling and automation scripts/applications in Python and/or .NET.
• Shared libraries/modules: Develop and maintain reusable libraries and modules for use by data engineers and analysts.
• Platform management automation: Design and implement automated processes for cost tracking, governance, resource provisioning, and monitoring of the platform.
• Collaboration: Work closely with data engineers, analysts, and security/infrastructure teams to turn requirements into platform features and capabilities.
• Troubleshooting: Diagnose and resolve issues related to infrastructure, pipelines, integrations (including networking, authentication, and connectivity problems).
• Documentation & standards: Contribute to documentation and the development of best practices and standards for the platform.

Strong hands-on experience with Microsoft Azure infrastructure (including Entra ID, Key Vault, networking, and storage).

• Practical experience with Azure Databricks (including Unity Catalog and cluster/job management).
• Proficiency in Terraform for Infrastructure as Code, preferably in multi-environment setups.
• Proficient in Python; working knowledge of SQL; .NET experience is a plus.
• Experience building and maintaining CI/CD pipelines (with Azure DevOps, GitHub Actions, or similar tools).
• Understanding of data platform governance and cost management.
• Comfortable working at the intersection of infrastructure and data engineering.

Key Technical Skills:

1. CI/CD (Azure, DevOps, GitHub)
2. Azure Databricks (Unity Catalog, cluster/job management)
3. Microsoft Azure infrastructure