Lead Data Engineer
Big Red Recruitment
Atherstone, United Kingdom
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
Senior Compensation
£ 80KJob location
Remote
Atherstone, United Kingdom
Tech stack
Azure
Cloud Computing
Data Integration
ETL
Data Warehousing
Data Processing
PySpark
Data Management
Databricks
Job description
- Lead the design and implementation of a new Databricks-based data warehousing solution
- Design and develop data models, ETL pipelines, and data integration processes
- Conduct large scale data processing using PySpark
- Monitor, tune, and optimize data platforms for reliability and performance
- Upskill the wider team in Databricks best practices and modern architecture patterns
Technologies:
- Architect
- Azure
- Databricks
- ETL
- PySpark
- Cloud
Requirements
- Proficiency in Azure Databricks and PySpark
- Experience in designing and developing data models and ETL pipelines
- Strong understanding of data integration processes
- Knowledge of monitoring, tuning, and optimizing data platforms
- Ability to lead and upskill teams in best practices and architecture patterns
- Experience with large scale data processing
- Previous experience in a Data Engineer or Architect role
- Permanent residency in the UK
