> Markdown version of [/jobs/ext/4628638-data-engineer](https://www.wearedevelopers.com/jobs/ext/4628638-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** SFE PARTNERS LLC
- **Location:** Atlanta, GA, United States
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Amazon Web Services, Amazon S3, Data Analysis, Big Data, BigQuery, Cloud Computing, Databases, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, IBM DB2, IBM InfoSphere DataStage, Apache Hadoop, Hadoop Distributed File System, Apache Hive, Python (Programming Language), Machine Learning, Mainframes, Scrum Methodology, Power BI, DataOps, Sqoop, Data Streaming, Tableau (Software), Teradata SQL, Unstructured Data, Data Ingestion, Apache Spark, Git, Data Lakes, Pyspark, Apache Kafka, Spark Streaming, Spotfire, Data Pipelines, TIBCO (Software), Databricks
- **Published:** November 7, 2026
- **Apply:** https://www.careerjet.com/jobad/us50370156cb0720366efe1ffc7109cb2c

## About the Role

* Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)
 * Python and PySpark
 * Spark SQL and Apache Spark
 * ETL/Data Pipeline Development
 * Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
 * Strong SQL querying and data modeling
 * Data ingestion, transformation, cleansing, and validation
 * Cloud platforms (AWS/GCP)
 * Data warehouse and large-scale data processing experience
 * Experience with streaming data using Kafka and Spark Streaming
 * Agile methodologies (Scrum, Kanban, SAFe)

Preferred Skills:

 * Scala / Scala-Spark
 * IBM DataStage, Tibco, or similar ETL tools
 * AWS services: S3, Glue, Lambda, Kinesis/MSK
 * BI Tools: Tableau, Power BI, Spotfire, OAC
 * Databases: Teradata, DB2, BigQuery, Mainframe
 * Git version control
 * Data governance and retention modeling

## Description

* Design, develop, and maintain scalable data pipelines and ETL solutions.
 * Build and optimize data ingestion frameworks for structured and unstructured data.
 * Implement data quality, validation, standardization, and governance processes.
 * Develop data infrastructure integrating multiple heterogeneous data sources.
 * Support analytics, reporting, machine learning, and business intelligence initiatives.
 * Work with Data Scientists and BI teams to prepare and curate data for analytics.
 * Develop scalable solutions for Big Data processing and streaming workloads.
 * Analyze data and provide actionable insights and recommendations to business stakeholders.
 * Gather requirements from business teams and translate them into technical solutions.
 * Lead and mentor junior data engineers.
 * Collaborate across multiple projects as the Data Engineering SME.
 * Drive adoption of Data Engineering best practices and standards.