Azure Data Engineer

TechAGI Solutions is seeking an experienced Azure Data Engineer to design, develop, test, and maintain scalable data pipelines and data management systems within the Microsoft Azure ecosystem. This role will collaborate with data scientists, analysts, BI developers, and software engineers to support advanced analytics and business intelligence initiatives., * Design, build, test, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

• Architect cloud-native data solutions using Azure Data Lake Storage Gen2, Azure SQL Database, and Azure Cosmos DB.
• Develop and optimize logical and physical data models for data warehouses and data lakes.
• Implement data validation, cleansing, monitoring, and governance processes to ensure data accuracy, integrity, and security.
• Troubleshoot and optimize data pipelines, database performance, and query execution for efficient, cost-effective processing.
• Gather requirements and translate business needs into reliable data solutions.
• Implement CI/CD pipelines using Azure DevOps, including automated testing and infrastructure as code.

• Bachelor’s degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field.
• Minimum 8 years of professional data engineering experience, including substantial hands-on experience with Microsoft Azure.
• Deep expertise in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage Gen2.
• Advanced SQL skills for complex querying and data manipulation.
• Strong programming skills in Python and/or Scala.
• Experience designing data warehouses, data lakes, and analytical data models.
• Experience with Azure DevOps, CI/CD, automated testing, and infrastructure as code.
• Strong troubleshooting, communication, and cross-functional collaboration skills.

Key Skills: Azure Data Engineering, Azure Data Factory, ADF, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage Gen2, ADLS Gen2, Azure SQL Database, Azure Cosmos DB, SQL, Python, Scala, ETL, ELT, Data Modeling, Data Warehousing, Data Quality, Data Governance, Azure DevOps, CI/CD, Infrastructure as Code