Senior Azure Databricks Engineer

This opportunity offers the chance to design, build, and support enterprise data platforms in a complex environment focused on analytics, AI initiatives, and project delivery. The role combines hands-on data engineering with platform administration across Microsoft Fabric, Databricks, Delta Lake, and OneLake. This position will play a key part in delivering scalable, secure, and reliable data solutions while supporting governance, performance optimisation, and operational excellence.

• Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Engineering, or a related discipline, or equivalent relevant experience \n, * 7-10 years of relevant experience in data engineering, data platforms, analytics, and cloud technologies \n

• Minimum 3+ years of hands-on experience with Databricks and 2+ years working in Microsoft Fabric environments \n

• Proven experience designing and supporting production-grade ETL/ELT and data integration solutions \n

• Experience administering enterprise-scale Databricks and Microsoft Fabric platforms \n

• Strong experience working with Azure data services and cloud-native architectures \n

• Demonstrated experience in stakeholder engagement, solution ownership, and technical leadership \n, * Strong expertise in Databricks, Spark, Delta Lake, Microsoft Fabric, OneLake, Lakehouse, and Data Warehouse architectures \n

• Advanced proficiency in Python, PySpark, SQL, and notebook-based development \n

• Experience designing and implementing ETL/ELT, data ingestion, and data transformation pipelines for structured and unstructured data sources \n

• Knowledge of batch processing, near real-time and streaming data ingestion from databases, APIs, files, cloud storage, and enterprise applications \n

• Strong understanding of dimensional modelling, data warehousing, semantic data structures, and data marts \n

• Experience implementing data quality frameworks, validation rules, reconciliation controls, and exception management processes \n

• Ability to optimise Spark workloads, SQL queries, storage strategies, partitioning, and distributed data processing performance \n

• Knowledge of Databricks workspace administration and Microsoft Fabric administration, including workspaces, capacities, Lakehouses, Data Warehouses, Dataflows, and Pipelines \n

• Experience with Microsoft Entra ID, RBAC, access controls, service principals, identity integrations, secrets management, key vaults, and secure data access \n

• Knowledge of platform monitoring, observability, troubleshooting, incident management, workload performance, and capacity planning \n

• Experience with Git, Azure DevOps, CI/CD pipelines, release management, automated testing, and deployment standards \n

• Understanding of metadata management, data lineage, cataloguing, governance processes, and compliance controls \n

• Ability to communicate technical solutions, trade-offs, risks, dependencies, and delivery plans to technical and non-technical stakeholders \n

• Ability to balance performance, scalability, reliability, security, and cost considerations in a fast-paced enterprise environment \n

• Knowledge of Infrastructure as Code, automation frameworks, and deployment templates \n

• Experience implementing enterprise data quality and reliability frameworks \n

• Experience with Fabric capacity optimisation, Databricks cluster optimisation, workload management, and FinOps practices \n

• Microsoft Certified: Fabric Analytics Engineer Associate \n

• Microsoft Certified: Azure Data Engineer Associate \n

• Databricks Certified Data Engineer Professional \n

• Databricks Certified Data Engineer Associate \n

\n

Airswift is an international workforce solutions provider within the energy, process and infrastructure industries. Airswift serves as a strategic partner to clients, offering a turnkey workforce solution to capture and deliver the top talent needed to complete successful projects by aligning with unique project needs. With over 1000 employees and 9,000 contractors operating in over 60 countries, Airswift’s geographical reach and pool of talent available is unmatched in the industry.