Databricks Technical Lead

17918
Hurley, United Kingdom
4 days ago
Apply on www.apply4u.co.uk
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
£260,000
Working hours
Regular working hours

Tech stack

Artificial Intelligence Business Logic Automation of Tests Microsoft Azure Code Generation Continuous Integration Information Engineering Dimensional Modeling Python (Programming Language) SAP ERP Power BI Azure Data Lake
+14 more
Reverse Engineering Salesforce.Com SAP (Applications) SQL Databases Azure Data Factory Large Language Models Agentic-AI Data Lakes Pyspark SAP S/4HANA Virtual Agents Terraform Data Pipelines Databricks

Job description

Databricks Technical Lead l Contract 12 Months Outside IR35 l Up to £1,000 per day Remote first, occasional on-site (Midlands & South East) The role Method Resourcing have been asked by one of our consulting partners to find a hands-on Databricks Technical Lead to design and deliver a new Azure Databricks platform for a major life sciences organisation replacing its enterprise data and analytics stack. The programme uses AI and LLM tooling to reverse-engineer existing pipelines, business logic and reports into documented, tested code. You’ll mentor three senior data engineers, own the technical architecture and stay hands-on in development. What you’ll do Architect and build the platform on Azure Databricks (Delta Lake, Unity Catalog, medallion architecture) Use AI agents and LLMs to reverse-engineer legacy reports, queries and transformations, and build repeatable AI-assisted processes for code generation, testing and migration Build automated testing and reconciliation to validate

Requirements

migrated data against existing reporting Integrate SAP, Azure Data Lake, Salesforce and other systems, and design gold-layer models for Power BI Set engineering standards (Workflows, CI/CD, IaC, governance) and mentor the team through to UAT and cutover What you’ll need Extensive hands-on experience delivering and technically leading enterprise solutions on Azure Databricks Demonstrable use of AI, LLMs or agents to reverse-engineer data pipelines or reporting logic Strong Python, PySpark and SQL Deep knowledge of Delta Lake, Unity Catalog, Workflows and medallion architecture Automated validation, reconciliation and testing experience Azure data engineering (ADLS Gen2, ADF) Databricks CI/CD (Terraform, Asset Bundles or Azure DevOps) SAP ECC or S/4HANA integration experience Dimensional modelling and Power BI semantic model familiarity Proven team leadership while staying hands-on Desirable Databricks certification (Data Engineer Professional or Solutions Architect) Platform migration, carve-out or separation experience Reusable AI agent frameworks for data engineering Strong governance and data quality experience Consulting background Contract outside IR35 £900-£1,000 per day Remote with some travel required 12 month contract If this is you, send your CV to and if suitable, i’ll be in touch. As you can appreciate, this role will receive a high volume of candidates so I will do my best to respond if possible. Databricks Technical Lead l Contract 12 Months Up to £1,000 per day Remote first, occasional on-site (Midlands & South East) RSG Plc is acting as an Employment Business in relation to this vacancy. WKCL1_UKTJ

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.apply4u.co.uk
Prepare application

Good distractions

Loading talks and stories from around this role…