> Markdown version of [/jobs/ext/2986407-data-engineer-python-pyspark-databricks](https://www.wearedevelopers.com/jobs/ext/2986407-data-engineer-python-pyspark-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer Python / PySpark / Databricks - **Company:** PropelSys Technologies LLC - **Location:** Bolingbrook, IL, United States (Remote available) - **Experience:** Expert - **Contract:** Temporary contract - **Skills:** Amazon Web Services, Data Analysis, Microsoft Azure, Program Optimization, Information Systems, Computer Programming, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Integration, Extract Transform Load (ETL), Data Warehousing, DevOps, Identity and Access Management, Python (Programming Language), Power BI, DataOps, SQL Stored Procedures, SQL Databases, Data Streaming, Tableau (Software), Google Cloud, Apache Spark, Build Management, Data Lakes, Pyspark, Kubernetes, Information Technology, Apache Kafka, Data Management, Machine Learning Operations, Data Lakehouse, Cloud Integration, Stream Processing, Data Pipelines, Databricks - **Published:** September 18, 2026 - **Apply:** https://www.dice.com/job-detail/893e6a12-1f23-48b9-b3dd-92c4ded2b8aa ## About the Role * Strong hands-on programming experience and ability to build solutions independently. * Deep experience with Python and PySpark Mandatory. * Working experience with Databricks. * Strong experience with stored procedures and legacy-to-modern migration. * Strong understanding of data and ability to interpret and transform data. * Healthcare data, clinical quality measures, or rule development experience strongly preferred., * 8+ years of data engineering and data platform experience. * 3+ years of hands-on Databricks architecture experience. * Strong expertise in Apache Spark, PySpark, SQL, and Delta Lake. * Experience with Azure Databricks, AWS Databricks, or Google Cloud Platform Databricks. * Strong expertise in Google Cloud Platform Pub/Sub, Azure Event Hubs, AWS Kinesis, or Apache Kafka. * Experience with producer/consumer architecture decisions and trade-offs. * Knowledge of data lakehouse architecture and data governance. * Experience with data integration tools and orchestration frameworks. * Strong understanding of security, access management, and compliance requirements. Preferred Skills * Databricks Certified Data Engineer Professional or equivalent. * Experience with Unity Catalog, Delta Live Tables, and Databricks Workflows. * Experience with Power BI, Tableau, or other BI platforms. * Knowledge of DataOps, DevOps, and MLOps practices. Education * Bachelor's degree in Computer Science, Engineering, Information Systems, or related field. * Master's degree preferred. ## Description * Design and build new tools and engines using Python, PySpark, and Databricks. * Maintain, upgrade, and migrate existing engines, including legacy stored procedures. * Develop and enhance data pipelines and automation workflows. * Analyze and process source data and modify or transform engine logic as required. * Support quality measure and rule development, including clinical measures. * Collaborate with onshore and offshore teams., The Databricks Architect will be responsible for designing, implementing, and governing enterprise-scale data platforms on Databricks. The role focuses on data architecture, cloud integration, data engineering best practices, analytics enablement, and platform optimization across Azure, AWS, or Google Cloud Platform., * Design and implement scalable data lakehouse architectures using Databricks. * Define enterprise data models, governance standards, and security controls. * Lead migration of legacy data warehouses and ETL workloads to Databricks. * Architect batch and real-time data processing solutions using Spark. * Optimize performance, scalability, reliability, and cost efficiency. * Collaborate with business stakeholders, data engineers, data scientists, and cloud architects. * Establish CI/CD, monitoring, and operational best practices. * Provide technical leadership and mentor engineering teams. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)