> Markdown version of [/jobs/ext/1219737-data-engineer](https://www.wearedevelopers.com/jobs/ext/1219737-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** HAVI - **Location:** Chicago, IL, United States - **Experience:** Experienced - **Salary:** $100,000.0 - $115,000.0 - **Contract:** Permanent contract - **Skills:** Microsoft Access, Agile Methodology, Artificial Intelligence, Microsoft Azure, Big Data, Code Review, Information Systems, Continuous Integration, Data Discovery, Information Engineering, Data Governance, Data Integration, Extract Transform Load (ETL), Data Masking, Data Transformation, Data Structures, Software Debugging, DevOps, Github, Apache Hive, Information Sciences, Log Analysis, Operational Databases, Performance Tuning, Role-Based Access Control, DataOps, Azure Data Lake, Scala (Programming Language), Data Streaming, Azure Data Factory, Autoscaling, Apache Spark, Git, Pytest, Data Lakes, Pyspark, Integration Tests, Information Technology, Bicep, Data Management, Database Replication, Virtual Agents, Api Design, Terraform, Data Pipelines, Key Vault, Databricks - **Published:** July 9, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=1f7eb8898b893b80 ## About the Role Bachelor's degree in computer science, data management, information systems, information science or a related field; advanced degree in computer science, data management, information systems, information science or a related field preferred. 3+ years in data engineering building production data pipelines (batch and/or streaming) with Spark on cloud. 2+ years hands-on Azure Databricks (PySpark/Scala, Spark SQL, Delta Lake) including: Delta Lake operations (MERGE/CDC, OPTIMIZE/Z-ORDER, VACUUM, partitioning, schema evolution). Unity Catalog (RBAC, permissions, lineage, data masking/row-level access). Databricks Jobs/Workflows or Delta Live Tables. Azure Data Factory for orchestration (pipelines, triggers, parameterization, IRs) and integration with ADLS Gen2, Key Vault. Strong SQL across large datasets; performance tuning (joins, partitions, file sizing). Data quality at scale (e.g., Great Expectations/Deequ), monitoring and alerting; debug/backfill playbooks. DevOps for data: Git branching, code reviews, unit/integration testing (pytest/dbx), CI/CD (Azure DevOps/GitHub Actions). Infrastructure as Code (Terraform or Bicep) for Databricks workspaces, cluster policies, ADF, storage. Observability & cost control: Azure Monitor/Log Analytics; cluster sizing, autoscaling, Photon; cost/perf trade-offs. Proven experience collaborating with cross-functional stakeholders (analytics, data governance, product, security) to ship and support data products. Dimensions & Stakholders: Content scope: Data engineering, data modeling, Agentic AI and automation, and data solution operationalization ## Description Responsible for working with the data management, data science, decision science, and technology teams to address supply chain data needs in demand and supply planning, replenishment, pricing, and optimization Develop/refine the data requirements, design/develop data deliverables, and optimize data pipelines in non-production and production environments Design, build, and manage/monitor data pipelines for data structures encompassing data transformation, data models, schemas, metadata, and workload management. The ability to work with both IT and business Integrate analytics and data science output into business processes and workflows Build and optimize data pipelines, pipeline architectures, and integrated datasets. These should include ETL/ELT, data replication/CI-CD, API design, and access Work with and optimize existing ETL processes and data integration and preparation flows and help move them to production Work with popular data discovery, analytics, and BI and AI tools in semantic-layer data discovery Adept in agile methodologies and capable of applying DevOps and DataOps principles to data pipelines to improve communication, integration, reuse, and automation of data flows between data managers and data consumers across the organization Implement Agentic AI capability to drive efficiency and opportunity, HAVI does not accept agency resumes submitted by third-party vendors unless a valid agreement has been signed and the HAVI Talent Acquisition Team has granted authorization for submissions for a specified position. Please do not submit or forward resumes to our site, HAVI employees, or any other company location. HAVI is not responsible for any fees related to unsolicited resumes. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python) - [Back(end) to the Future: Embracing the continuous Evolution of Infrastructure and Code](https://www.wearedevelopers.com/videos/440-back-end-to-the-future-embracing-the-continuous-evolution-of-infrastructure-and-code) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Automagic Configuration in Python](https://www.wearedevelopers.com/videos/363-automagic-configuration-in-python) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)