Azure Data Engineer

• Develop, construct, test and maintain data architectures within large scale data processing systems.
• Develop and manage data pipelines using Azure Data Factory, Delta Lake and Spark.
• Utilise Azure Cloud architecture knowledge to design and implement scalable data solutions.
• Utilise Spark, SQL, Python, R, and other data frameworks to manipulate data and gain a thorough understanding of the dataset's characteristics.
• Interact with API systems to query and retrieve data for analysis.
• Collaborate with business users / stakeholders to gather and agree requirements.

To be considered for the post you'll need at least 5 years experience ideally with 1 or 2 years at a senior / lead level. You'll need to be goal driven and able to take ownership of work tasks without the need for constant supervision. You'll be engaging with multiple business areas so the ability to communicate effectively to understand requirements and build trusted relationships is a must. It's likely you'll have most, if not all the following:

• Experience as a Senior Data Engineer or similar
• Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD.
• Experience with PySpark, Python, SQL and other data engineering development tools.
• Experience with metadata driven pipelines and SQL serverless data warehouses.
• Knowledge of querying API systems.
• Experience building and optimising ETL pipelines using Databricks.
• Strong problem-solving skills and attention to detail.
• Understanding of data governance and data quality principles.
• A degree in computer science, engineering, or equivalent experience.

Salary will be dependent on experience and likely to be in the region of £70,000 - £80,000 although client may consider higher for outstanding candidate. Our client can also provide a vibrant, rewarding, and diverse work environment that supports career development.