Data Platform Engineer

This role is focused on platform engineering and operations, NOT on building data pipelines or ML models. You will work on improving, automating, and scaling the platform itself., Platform Engineering & Operations

• Improve implementation patterns across jobs and platform components
• Identify and resolve scalability, reliability, and maintainability bottlenecks
• Standardise workspace configurations and environment structures
• Monitor platform health and performance

Unity Catalog Management

• Maintain consistent and secure Unity Catalog configurations
• Ensure governance structures (catalogs, schemas, grants, service principals) are efficient and developer-friendly

Performance & Cost Optimisation

• Optimise compute configurations and autoscaling behaviour
• Analyse Spark jobs, Delta Lake usage, and storage layout for performance and cost efficiency

Automation & Infrastructure-as-Code

• Automate environments using Databricks Asset Bundles (DABs) and Terraform
• Ensure reproducible, versioned environments aligned with CI/CD principles
• Minimise manual configuration

Documentation & Guidance

• Document platform patterns and operational runbooks
• Support engineers and analysts working with Databricks

• 3-5+ years in Platform (Data) Engineering
• Strong hands-on experience with Azure Databricks
• Strong knowledge of Unity Catalog (access models, governance, service principals, workspace configs)
• Solid PySpark and SQL skills
• Understanding of performance optimisation (partitioning, Z-ordering, caching, data layout)
• Experience with CI/CD, Git workflows, and Infrastructure-as-Code
• Bonus: MLOps experience