Databricks Engineer

Gazelle Global Consulting
London, United Kingdom
2 days ago
Apply on www.totaljobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Amazon Web Services Amazon S3 Cloud Computing Continuous Integration Data Architecture Identity and Access Management SQL Databases Data Streaming Data Processing Git Data Lakes Pyspark
+2 more
Data Pipelines Databricks

Job description

  • Build and maintain Databricks Lakehouse platforms across development, testing and production environments.
  • Develop data pipelines using PySpark, SQL and Delta Lake.
  • Configure Unity Catalog, permissions, access controls and security policies.
  • Implement CI/CD pipelines and automate platform deployments.
  • Manage Databricks Jobs, Workflows and Declarative Pipelines.
  • Integrate Databricks with AWS services and cloud infrastructure.
  • Support batch and incremental data processing.
  • Optimise platform performance, monitoring and operational reliability.

Requirements

  • Strong hands-on experience with Databricks and PySpark.
  • Excellent knowledge of Delta Lake and Lakehouse architecture.
  • Experience with Unity Catalog, Autoloader and Structured Streaming.
  • Knowledge of AWS S3, IAM, KMS, Glue and Lambda.
  • Experience with Git, CI/CD and data pipeline development.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.totaljobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…