> Markdown version of [/jobs/ext/4095430-data-engineer](https://www.wearedevelopers.com/jobs/ext/4095430-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** The Client
- **Location:** Atlanta, GA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Tibco Ems, Microsoft Access, Agile Methodology, Amazon Web Services, Amazon S3, Big Data, Cloud Computing, Information Systems, Databases, Data Architecture, Data Validation, Information Engineering, Data Governance, Extract Transform Load (ETL), Database Queries, Apache Hive, IBM WebSphere MQ, Identity and Access Management, Iterative and Incremental Development, Python (Programming Language), Machine Learning, Enterprise Messaging Systems, NoSQL, Performance Tuning, Scrum Methodology, SQL Databases, Systems Integration, Unstructured Data, Data Ingestion, Snowflake, Apache Spark, Data Lakes, Pyspark, Information Technology, Cassandra, Data Analytics, Apache Kafka, Operational Systems, Data Pipelines, Databricks
- **Published:** October 9, 2026
- **Apply:** https://www.dice.com/job-detail/66ed83b7-0eec-4c91-a0cd-ba73d2046218

## About the Role

* 5+ years of professional data engineering experience working with large datasets in production.
 * Hands-on Databricks experience, including Jobs/Workflows, notebooks, and production pipelines.
 * Hands-on Delta Lake experience, including incremental processing, merges/upserts, schema evolution, and performance optimization.
 * Hands-on experience designing and operating Delta Live Tables (DLT) pipelines.
 * Practical Unity Catalog or comparable Databricks governance experience.
 * 4+ years of Apache Spark engineering experience using Spark SQL and/or PySpark.
 * 3+ years of Apache Kafka or managed Kafka experience, including high-volume event processing, scaling, lag, and replay.
 * 3+ years of AWS experience supporting data and analytics platforms, including S3, IAM, Glue, Lambda, or MSK.
 * Strong SQL skills, including intermediate-to-advanced query development and optimization.
 * Experience building and operating ETL/ELT pipelines in a Databricks Lakehouse environment.
 * Experience working in an Agile environment such as Scrum, Kanban, or SAFe.

Preferred Experience

 * Snowflake or other large-scale analytical databases.
 * NoSQL databases such as Cassandra.
 * Enterprise messaging technologies such as TIBCO EMS or IBM MQ.
 * Integrating operational system feeds into cloud-based analytics and Lakehouse environments.
 * Supporting BI, reporting, analytics, or machine learning initiatives.

Education

 * Bachelor's degree in Information Systems, Computer Science, Computer Information Systems, or a related field preferred.
 * Equivalent practical experience may be considered.

## Description

Client Corporation is seeking a Senior Data Engineer to design, build, and operate scalable data solutions within a Databricks-based Lakehouse and streaming analytics environment. This role partners with business, BI, data science, and data modeling teams to translate requirements into production-ready data pipelines and curated datasets for reporting, analytics, and machine learning., * Partner with business, BI, data science, and data modeling teams to define requirements and deliver scalable data solutions.

 * Design, develop, and operate production-grade data ingestion, transformation, and integration pipelines.
 * Build and maintain Databricks, Spark, Delta Lake, and Delta Live Tables (DLT) pipelines supporting batch and streaming workloads.
 * Integrate and transform structured and unstructured data using Databricks, SQL, Python, Spark, AWS, and Kafka.
 * Implement incremental processing, merges/upserts, schema evolution, data quality checks, and other Lakehouse best practices.
 * Apply data governance, security, access, retention, and sensitive-data handling practices in accordance with enterprise standards.
 * Establish testing, data quality, observability, and operational readiness practices.
 * Troubleshoot and optimize pipelines for performance, reliability, scalability, cost, and maintainability.
 * Contribute to data architecture and engineering standards and support initiatives from requirements through deployment and ongoing operations.
 * Participate in Agile delivery, including backlog refinement, iterative development, and dependency coordination.