> Markdown version of [/jobs/ext/3080892-aws-databricks-engineer](https://www.wearedevelopers.com/jobs/ext/3080892-aws-databricks-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AWS Databricks Engineer - **Company:** Virtualan Software LLC - **Location:** Chicago, IL, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Amazon S3, Cloud Database, Databases, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Systems, Data Warehousing, Identity and Access Management, Python (Programming Language), Performance Tuning, Cloud Services, SQL Databases, Management of Software Versions, Data Ingestion, Apache Spark, Software Troubleshooting, AWS Lambda, Git, Data Lakes, Pyspark, Integration Tests, Information Technology, Deployment Automation, AWS Glue, Apache Kafka, Video Streaming, Terraform, Data Pipelines, Amazon Elastic Mapreduce (EMR), Amazon Redshift, Databricks - **Published:** September 25, 2026 - **Apply:** https://www.dice.com/job-detail/7b2ba347-fd1c-46b6-935a-019cf9daabfb ## About the Role * 5+ years of experience in Data Engineering. * Strong hands-on experience with Databricks. * Strong knowledge of Apache Spark and PySpark. * Proficiency in Python and SQL. * Strong experience with AWS cloud services, particularly: + Amazon S3 + AWS Glue + AWS Lambda + Amazon Redshift + Amazon EMR + IAM * Experience with Delta Lake and Lakehouse architecture. * Strong understanding of ETL/ELT, data warehousing, and data lake concepts. * Experience developing production-grade data pipelines. * Experience with Git and CI/CD tools. * Strong troubleshooting and performance-tuning skills. Preferred Skills * Experience with Databricks Workflows/Jobs and Unity Catalog. * Experience with AWS Step Functions or Airflow. * Experience with real-time/streaming technologies such as Kafka or Kinesis. * Knowledge of Terraform or Infrastructure as Code. * Experience with data governance, lineage, and security. * Databricks or AWS certifications are a plus., Bachelor s degree in Computer Science, Information Technology, Engineering, or a related field preferred. ## Description We are seeking an experienced AWS Databricks Engineer to design, develop, and maintain scalable data engineering solutions using Databricks, Apache Spark, and AWS cloud services. The ideal candidate will have strong experience in data pipelines, ETL/ELT, data lake architecture, and cloud-based data processing., * Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark. * Build and optimize ETL/ELT workflows for batch and streaming data processing. * Develop solutions using Databricks notebooks, Delta Lake, PySpark, and SQL. * Implement and manage data solutions on AWS, including S3, Glue, Lambda, EMR, Redshift, and related services. * Develop and maintain data lake/lakehouse architectures using Databricks and AWS. * Perform data ingestion from databases, APIs, files, and other enterprise data sources. * Implement Delta Lake features such as schema evolution, partitioning, optimization, and data versioning. * Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability. * Implement security, access controls, data governance, and best practices across AWS and Databricks environments. * Work with data architects, analysts, developers, and business stakeholders to understand requirements and deliver data solutions. * Implement CI/CD and deployment automation for Databricks and data engineering workloads. * Develop unit/integration testing and ensure data quality and pipeline reliability. ## Related Videos - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)