Senior Data Engineer
Role details
Job location
Tech stack
Job description
As a Senior Data Engineer, you will play a key role in transforming raw data into powerful insights that drive strategic decisions across the organization. Leveraging modern technologies such as Azure, Terraform, Databricks, Kafka, and Power BI, you will design and implement complex ETL workflows and real-time streaming pipelines with a strong focus on IoT use cases. You will also contribute to building a state-of-the-art data lakehouse architecture using Delta Lake tables.
Your work will empower teams both onshore and offshore to access, analyze, and act on data more effectively than ever before. You'll collaborate closely with end users and stakeholders to deliver robust, scalable solutions aligned with IT best practices.
As a senior member of the team, you will coach and mentor others, champion engineering best practices, contribute to long-term architectural strategy, and take a forward-looking approach, anticipating challenges and identifying opportunities early to drive sustainable growth., * Design and implement data pipelines using Azure services, with a strong focus on Databricks;
- Develop and maintain real-time data streaming pipelines to process IoT data, using tools such as Kafka, Azure IoT Hub, and stream processing frameworks like Spark Streaming, Flink/PyFlink, or Azure Stream Analytics;
- Build and optimize a modern data lakehouse architecture, employing Delta Lake for efficient and reliable data storage and management;
- Create and maintain an environment for Power BI dashboards and reports that deliver actionable insights for business stakeholders;
- Translate business requirements into robust, scalable data solutions by working closely with cross-functional teams;
- Enforce data governance and quality standards to ensure the integrity, accuracy, and security of the data infrastructure;
- Proactively monitor, troubleshoot, and optimize data pipelines to meet evolving business needs;
- Design and develop scalable, reusable, and reliable ETL batch processing patterns.
Requirements
Are you a Senior Data Engineer with a passion for turning complex data into clear insights? Do you enjoy working hands-on with the Azure Data Stack and solving technical challenges with real world impact?, We're on the lookout for a self-starting, skilled, and meticulous Senior Data Engineer who thrives in a collaborative environment. You have a strong foundation in data engineering and a proven ability to build scalable, high-quality data solutions. Your expertise in automated testing of data pipelines will be critical in maintaining the integrity and performance of our data infrastructure.
You bring a strong understanding of data engineering fundamentals and experience with Azure Cloud, Databricks, and databases. You're passionate about automation and innovation, quick to adapt to change, and eager to continuously learn. With your proactive attitude and strong communication skills, you work seamlessly with colleagues across teams.
What you bring:
- An HBO or BSc degree in Computer Science, Engineering, Information Technology, or a related field;
- Proficiency in SQL and Python for data processing;
- Demonstrable experience in data modeling, warehousing, ETL, and real-time streaming;
- Strong problem-solving skills, with the ability to perform under pressure and meet deadlines;
- Excellent communication skills in English, with a history of collaborating in cross-functional teams;
- Experience working in Scrum/Agile environments, using tools like Jira or Azure DevOps;
- Hands-on experience with CI/CD pipelines and tools such as Git and Terraform.
