Data Engineer - SC Cleared

• Design, build and deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage.
• Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake.
• Champion strong data quality, governance and observability practices.
• Modernise Legacy systems and support large-scale migrations into Azure.
• Provide technical leadership, set engineering standards and contribute to architectural decisions.
• Mentor engineers and foster a culture of continuous improvement.
• Work closely with architecture, analytics and business teams to align engineering solutions with organisational needs.

• Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.
• Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.
• Proven experience in CI/CD practices using industry-standard tools (eg, GitHub Actions, Azure DevOps).
• Strong understanding of data architecture principles and cloud-native design patterns., * Demonstrated ability to lead technical delivery, mentor engineering teams and collaborate with stakeholders to ensure alignment between data solutions and business strategy.
• Proficiency in Linux/Unix environments and Shell Scripting.
• Deep understanding of source control, testing strategies, and agile development practices.
• Self-motivated with a strategic mindset and a passion for driving innovation in data engineering., * Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives.
• Familiarity with:

• Apache Airflow
• Data modelling and metadata management

• Experience influencing enterprise data strategy and contributing to architectural governance.

Guidant, Carbon60, Lorien & SRG - The Impellam Group Portfolio are acting as an Employment Business in relation to this vacancy.