Data Engineer

Qualient Technology Solutions UK Limited

London, UK

5 days ago

Apply on www.careerboard.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

3 years minimum

Working hours

Regular working hours

Job source

www.careerboard.com

Tech stack

Agile Methodology Microsoft Azure Big Data Cloud Storage Data Architecture Information Engineering Data Governance Data Warehousing DevOps Python (Programming Language) SQL Azure NoSQL

+15 more

Power BI SQL Databases Data Streaming Tableau (Software) Azure Data Factory Apache Spark Git Pyspark Kubernetes Apache Kafka Software Version Control Data Pipelines Docker Jenkins Databricks

Requirements

10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks.
Strong proficiency in Python and Spark (PySpark) or Scala.
Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns.
Extensive experience with Azure data services, including Azure Data Factory, Azure Blob Storage, and Azure SQL Database.
Experience working with large datasets and complex data pipelines.
Experience with data architecture design and data pipeline optimization.
Proven expertise with Databricks, including hands-on implementation experience and certifications.
Experience with SQL and NoSQL databases.
Experience with data quality and data governance processes.
Experience with version control systems (eg, Git).
Experience with Agile development methodologies.
Excellent communication, interpersonal, and problem-solving skills.
Experience with streaming data technologies (eg, Kafka, Azure Event Hubs).
Experience with data visualisation tools (eg, Tableau, Power BI).
Experience with DevOps tools and practices (eg, Azure DevOps, Jenkins, Docker, Kubernetes).
Experience working in a financial services or economic data environment.
Azure certifications related to data engineering (eg, Azure Data Engineer Associate).