Senior Data Engineer Python, Databricks - CER Financial
cer Financial Limited
yesterday
Role details
Contract type
Temporary to permanent Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
Senior Compensation
£ 234KJob location
Tech stack
Airflow
Amazon Web Services (AWS)
Azure
Big Data
Google BigQuery
Cloud Computing
Continuous Integration
Information Engineering
ETL
Data Warehousing
Python
System Availability
Snowflake
Spark
Data Lake
PySpark
Kubernetes
Kafka
Data Pipelines
Docker
Databricks
Job description
- Design, build and optimise scalable data pipelines using Python and Databricks.
- Develop ETL/ELT workflows with Spark and Delta Lake.
- Build and maintain Lakehouse data models and architectures.
- Ensure data quality, governance and platform reliability.
- Optimise large-scale data processing and support analytics and ML initiatives.
- Mentor junior engineers and promote engineering best practices.
Requirements
- 10+ years' experience in Data Engineering.
- Strong Python and Databricks expertise, including Spark/PySpark and Delta Lake.
- Experience building scalable ETL pipelines and working with SQL.
- Knowledge of cloud platforms (AWS, Azure or GCP).
- Experience with modern data warehouses such as Snowflake, Redshift or BigQuery.
- Familiarity with Airflow or Databricks Workflows.
- Databricks certifications and experience with Kafka, Docker, Kubernetes or CI/CD are advantageous.