Junior Data Engineer

• Develop, test, and maintain scalable data pipelines using Python and PySpark within an Azure Databricks development environment
• Clean, aggregate, and transform complex datasets to meet business requirements.
• Assist with data quality checks and support data validation requests.
• Work alongside different teams/departments to understand their data needs and provide engineered solutions.
• Assist in monitoring pipeline performance and troubleshooting data processing issues.

We are seeking a motivated and detail-oriented Junior Data Engineer to join our clients growing data team. In this role, you will help build, transform and maintain reliable data pipelines that support their data architecture. This is an excellent opportunity for a candidate with a strong foundation in Python and data transformation, where they'd be able to develop their expertise in data processing within a cloud-based data platform., * Strong proficiency in Python, particularly for data manipulation and automation scripts; experience in libraries such as pandas, PySpark, numpy, pyodbc or associated would be beneficial

• Hands-on experience (or academic project experience) using PySpark.
• An understanding of ETL/ELT processes and/or building data flow pipelines in any technology, and how to structure data for analytical or reporting use.
• Familiarity with SQL, writing queries, joins, and subqueries to interact with relational databases.
• A proactive approach to debugging code and identifying data inconsistencies.
• 0-2 years hands on experience in a data engineering/analytics role or equivalent academic/project experience.

Desired Skills:

• Experience with Databricks or similar cloud-based data platforms like Azure, AWS, or GCP.
• Exposure to Git workflows, CI/CD pipelines, or source control principles more generally.
• Exposure to tools like ADF, Synapse or Fabric is a plus.
• Medallion data architecture architecture: Understanding of bronze, silver, gold layers within a medallion data architecture.
• A degree in Computer Science, Data Engineering, Information Systems, or a related technical field.