Senior Azure Data Engineer (ID: 3907)

• Design, develop, and maintain scalable, high-performance data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Build and optimize large-scale data processing solutions using PySpark and Databricks to support enterprise analytics and business intelligence initiatives.
• Develop robust Python-based ETL/ELT solutions for extracting, transforming, and loading data from diverse data sources.
• Collaborate with architects, data analysts, business stakeholders, and engineering teams to understand data requirements and deliver scalable cloud-based data solutions.
• Implement and maintain CI/CD pipelines using Azure DevOps to automate testing, deployment, and release management of data engineering solutions.
• Monitor, troubleshoot, and optimize Azure data pipelines to ensure reliability, scalability, and operational excellence.
• Ensure data quality, governance, security, and compliance across the enterprise data platform.
• Optimize SQL queries and data processing workflows to improve system performance and efficiency.
• Participate in Agile ceremonies and contribute to continuous improvement of engineering processes and cloud data architecture.
• Stay current with emerging Azure technologies, cloud data engineering best practices, and modern data platform innovations., * Develop high-performance ETL pipelines using Python, Azure Data Factory, and Databricks.
• Build and optimize distributed data processing solutions using PySpark.
• Automate deployment processes through Azure DevOps CI/CD pipelines.
• Troubleshoot complex data processing and pipeline performance issues.
• Ensure enterprise data quality, governance, and security standards are maintained.
• Collaborate effectively with cross-functional teams to deliver business-driven data solutions.
• Continuously improve cloud data platforms by adopting modern engineering best practices.

What We Bring to the Table:

• Opportunity to work on enterprise-scale Azure cloud data engineering and analytics initiatives.
• Exposure to modern cloud-native technologies, big data platforms, and large-scale distributed data processing.
• Collaborative environment with experienced cloud architects, data engineers, and analytics professionals.
• Challenging projects involving data modernization, automation, and cloud transformation.
• Opportunities for continuous learning, Azure certifications, and technical career growth.
• A culture that encourages innovation, collaboration, and engin eering excellence.

• 6-8 years of experience in Data Engineering, Cloud Data Platforms, or Big Data Engineering.
• Strong hands-on expertise in Azure Data Factory (ADF) for data orchestration and workflow automation.
• Extensive experience with Azure Databricks for distributed data processing and analytics.
• Advanced programming skills in Python for ETL development, automation, and data engineering.
• Strong experience using PySpark for processing large-scale datasets and building scalable data pipelines.
• Proficiency in SQL for querying, optimizing, and managing large enterprise datasets.
• Strong understanding of ETL/ELT methodologies, data integration, and cloud-native data architectures.
• Experience working within Agile development environments.
• Excellent analytical, troubleshooting, problem-solving, and communication skills.