Data Analyst - Databricks

• Support the engineering, optimisation, and sustainability of the Authority's Databricks-based Lakehouse platform.
• Develop high-quality ETL pipelines and transformation processes to ensure accuracy, efficiency, and compliance.
• Reverse-engineer and rationalise existing Python-based pipelines developed by external partners.
• Document data flows, dependencies, and processes to enable effective BAU handover and maintainability.
• Provide technical assurance, mentoring, and clear documentation to build long-term internal capability.
• Implement best-practice data models (eg, medallion architecture, dimensional modelling).
• Ensure data structures support self-service analytics, BI reporting, and advanced analytics.
• Contribute to evolving architectural standards and enhance the enterprise data model.

1. Databricks expertise - notebooks, Delta Lake, Databricks Genie bots.
2. Strong ETL design, optimisation, and debugging experience.
3. Advanced Python for ETL development and automation.
4. Strong SQL for validation, transformation, and optimisation.
5. Experience with Azure Data Factory for ingestion and scheduling.
6. Proficiency with GitHub and collaborative development workflows.

Guidant, Carbon60, Lorien & SRG - The Impellam Group Portfolio are acting as an Employment Business in relation to this vacancy.