Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Compensation

£143,000.0

Working hours

Regular working hours

Job source

www.careerboard.com

Tech stack

Application Programming Interfaces (APIs) Microsoft Azure Information Engineering Extract Transform Load (ETL) Data Transformation Python (Programming Language) SQL Databases Data Ingestion Azure Data Factory Databricks

Job description

• Build and maintain data ingestion pipelines using Databricks.
• Connect to multiple data sources including SQL databases, APIs and Azure services.
• Develop pipelines using Azure Data Factory and Databricks Lakeflow.
• Implement appropriate ingestion strategies including incremental and batch loads.
• Develop ETL/ELT processes using Databricks Medallion Architecture.
• Write and maintain Python data engineering code.
• Use dbt for data transformation and modelling.
• Ensure pipelines are scalable, reliable and maintainable.

Requirements

• Strong commercial Databricks experience.
• Proven data ingestion experience.
• Azure Data Factory.
• Databricks Lakeflow.
• SQL and API integration.
• Medallion Architecture.
• Strong Python.
• dbt.
• Experience with incremental/batch data ingestion.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerboard.com

Prepare application

• Draft this with your agent
• Open in Claude
• Open in ChatGPT
• Copy prompt