Data Engineer w/ AI

The ideal candidate has strong Python skills, experience orchestrating workflows with Apache Airflow, and hands-on expertise working with modern data libraries such as Polars. You will work extensively with Docker and deploy data solutions in the Microsoft Azure cloud environment. This role will also focus on modernizing the data platform by migrating existing data pipelines from Azure Synapse Analytics to Apache Airflow-based orchestration frameworks., * Use Claude to assist in your delivery

• Design, develop, and maintain reliable, scalable data pipelines using Python, Polars, and Apache Airflow
• Maintain Airflow deployments in Docker to ensure portability and consistency across environments
• Deploy, manage, and monitor data solutions in Azure (e.g. Azure Container Instances, Azure Storage, Azure SQL)
• Optimize data performance, reliability, and cost in cloud-based architectures
• Collaborate with analytics, BI, and data science teams to ensure high-quality, well-modeled data
• Implement data quality checks, monitoring, and alerting
• Follow best practices for version control, testing, CI/CD, and infrastructure-as-code (e.g., GitHub, Azure DevOps, Azure Artefacts, Azure Container Registry, Terraform)
• Document data pipelines, system architecture, and operational procedures (Confluence and Miro)

• Claude usage
• Strong proficiency in Python for data engineering use cases
• Hands-on experience with Apache Airflow for workflow orchestration
• Practical knowledge of Polars for data processing and transformation
• Solid experience building and running Docker containers
• Experience working in Microsoft Azure cloud environments
• Understanding of data modeling and schemas, ETL/ELT patterns, and distributed data systems
• Experience with Git and collaborative development workflows
• Strong problem-solving skills and attention to detail, * Experience with Azure data services such as Azure Data Factory, Synapse Analytics, Databricks, or Event Hubs
• Experience deploying containerized workloads using AKS or similar orchestration platforms
• Familiarity with CI/CD pipelines for data platforms

• Opportunity to work on modern, cloud-native data platforms
• Collaborative, data-driven engineering culture
• Competitive compensation and benefits
• Professional growth and learning opportunities

Pay: From $95.00 per hour