> Markdown version of [/jobs/ext/4334445-sr-data-engineer](https://www.wearedevelopers.com/jobs/ext/4334445-sr-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Sr. Data Engineer - **Company:** Login Consulting Services Inc. - **Location:** Chicago, IL, United States (Remote available) - **Experience:** Expert - **Contract:** Temporary to permanent - **Skills:** Java (Programming Language), Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Amazon S3, Microsoft Azure, Batch Processing, Big Data, Unix, Cloud Storage, Continuous Integration, Data Validation, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Transformation, Data Warehousing, Relational Databases, Database Queries, Apache Hadoop, Hadoop Distributed File System, Apache Hive, Job Scheduling, Python (Programming Language), Performance Tuning, Scrum Methodology, Power BI, Standard Sql, Azure Data Lake, Scala (Programming Language), Shell Script, SQL Stored Procedures, SQL Databases, SQL Server Reporting Services, SQL Server Integration Services, Tableau (Software), Workflow Management Systems, Enterprise Data Management, Data Logging, Data Processing, Freeform SQL, Enterprise Software Applications, Data Ingestion, Azure Data Factory, Apache Spark, Git, Data Lakes, Pyspark, Apache Kafka, Tools for Reporting, Azure Synapse Analytics, Data Pipelines, Jenkins, Databricks, Control M - **Published:** October 18, 2026 - **Apply:** https://www.careerbuilder.com/job-details/sr-data-engineer-chicago-il--4c3ed372-a451-47c7-b98f-6bcf480c7834 ## About the Role while ensuring established performance and quality standards are met. Requirements: Essential Qualifications * 8+ years of experience in Big Data Development, Data Engineering, ETL/ELT development, and enterprise data integration. * Strong hands-on development experience with Python, PySpark, Apache Spark, SQL, and Databricks. * Experience designing and developing large-scale data processing applications and ETL/ELT pipelines. * Hands-on experience with Azure Data Factory (ADF) for data ingestion, transformation, integration, and workflow orchestration. * Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive. * Strong SQL development skills, including complex queries, stored procedures, transformations, and performance optimization. * Experience developing and maintaining ETL workflows using SSIS. * Experience with Unix/Linux Shell Scripting and enterprise job scheduling tools such as Control-M. * Strong understanding of Delta Lake, Databricks Workflows, data warehousing, data modeling, and source-to-target data processing. * Experience in data validation, reconciliation, error handling, production support, and performance tuning. * Experience working in Agile/Scrum environments with strong analytical, troubleshooting, communication, and collaboration skills. Bachelors degree required. Desired Qualifications * Experience with Azure cloud services, including ADLS Gen2, Azure Synapse Analytics, and Azure DevOps. * Experience with AWS services, including S3, Glue, EMR, Lambda, and Redshift. * Knowledge of Kafka, Airflow, Java/Scala, Git, Jenkins, and CI/CD pipelines. * Experience with Power BI, SSRS, Tableau, or other enterprise reporting and analytics tools. * Prior experience supporting large-scale enterprise data platforms in healthcare, insurance, or other regulated industries. U.S. Medicare Experience Big Data Development & ETL/ELT 6+ years Python, PySpark & Apache Spark 6+ years Databricks / Apache Spark / PySpark /Delta lakes 6+ years SQL & Data Transformation 6+ years Azure Data Factory (ADF) 5+ years Hadoop/HDFS & Hive 5+ Unix/Linux Shell Scripting & Control-M 5+ ## Description A leading Consulting firm is seeking a Sr. Data Engineer for a 6+ month contract to hire role. This position is full remote, but client is based in Chicago, IL. Mandatory skills: Databricks, ADF, Big Data, Hive, Hadoop, HDFS, Python, Pyspark Duties: * Design, develop, and maintain scalable Big Data applications, ETL/ELT pipelines, and enterprise data integration solutions. * Develop large-scale data processing solutions using Databricks, Apache Spark, PySpark, Python, Hadoop/HDFS, Hive, and Azure Data Factory (ADF). * Build and optimize Databricks notebooks, Spark jobs, Delta Lake tables, and Lakehouse data processing workflows. * Design and develop data ingestion, transformation, and integration pipelines using Azure Data Factory (ADF), including pipelines, activities, triggers, and dependencies. * Develop complex SQL queries, stored procedures, data transformations, and source-to-target mapping logic. * Build and maintain ETL workflows using SSIS and modern cloud-based data integration technologies. * Develop reusable Python, PySpark, and Unix/Linux Shell Scripts for data processing, automation, validation, and operational support. * Integrate data from relational databases, APIs, files, cloud storage, and enterprise applications. * Develop and manage batch processing, job scheduling, dependencies, and workflow orchestration using Control-M, ADF, and Databricks Workflows. * Implement data quality checks, reconciliation, exception handling, logging, and error recovery mechanisms. * Perform performance tuning and optimization of Spark jobs, Databricks workloads, SQL queries, and data pipelines. * Collaborate with data architects, business analysts, QA teams, and engineering teams to deliver scalable enterprise data solutions. * Participate in Agile/Scrum activities, including sprint planning, refinement, daily stand-ups, reviews, and retrospectives. * Support production deployments, troubleshooting, incident resolution, and post-deployment verification while ensuring established performance and quality standards are met. Requirements: Essential Qualifications * 8+ years of experience in Big Data Development, Data Engineering, ETL/ELT development, and enterprise data integration. * Strong hands-on development experience with Python, PySpark, Apache Spark, SQL, and Databricks. * Experience designing and developing large-scale data processing applications and ETL/ELT pipelines. * Hands-on experience with Azure Data Factory (ADF) for data ingestion, transformation, integration, and workflow orchestration. * Experience with Hadoop ecosystem technologies, including Hadoop/HDFS and Hive. * Strong SQL development skills, including complex queries, stored procedures, transformations, and performance optimization. * Experience developing and maintaining ETL workflows using SSIS. * Experience with Unix/Linux Shell Scripting and enterprise job scheduling tools such as Control-M. * Strong understanding of Delta Lake, Databricks Workflows, data warehousing, data modeling, and source-to-target data processing. * Experience in data validation, reconciliation, error handling, production support, and performance tuning. * Experience working in Agile/Scrum environments with strong analytical, troubleshooting, communication, and collaboration skills. Bachelors degree required. Desired Qualifications * Experience with Azure cloud services, including ADLS Gen2, Azure Synapse Analytics, and Azure DevOps. * Experience with AWS services, including S3, Glue, EMR, Lambda, and Redshift. * Knowledge of Kafka, Airflow, Java/Scala, Git, Jenkins, and CI/CD pipelines. * Experience with Power BI, SSRS, Tableau, or other enterprise reporting and analytics tools. * Prior experience supporting large-scale enterprise data platforms in healthcare, insurance, or other regulated industries. U.S. Medicare Experience Big Data Development & ETL/ELT 6+ years Python, PySpark & Apache Spark 6+ years Databricks / Apache Spark / PySpark /Delta lakes 6+ years SQL & Data Transformation 6+ years Azure Data Factory (ADF) 5+ years Hadoop/HDFS & Hive 5+ Unix/Linux Shell Scripting & Control-M 5+