Databricks Data Engineer
Robotics Technologies LLC
Atlanta, GA, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Compensation
$158,080
Working hours
Regular working hours
Job source
Tech stack
Amazon Web Services
Microsoft Azure
Cloud Storage
Computer Programming
Databases
Continuous Integration
Data Architecture
Data Validation
Data Governance
Extract Transform Load (ETL)
Data Security
Data Systems
+19 more
Data Warehousing
Apache Hive
Python (Programming Language)
Cloud Services
SQL Databases
Data Streaming
Workflow Management Systems
Data Processing
Apache Spark
Git
Data Lakes
Pyspark
Infrastructure Automation Frameworks
Apache Kafka
Machine Learning Operations
Terraform
Stream Processing
Data Pipelines
Databricks
Job description
- We are looking for a skilled Databricks Data Engineer to develop and manage enterprise-scale data processing solutions using Databricks, Apache Spark, PySpark, and Delta Lake.
- The ideal candidate should have strong experience in building data pipelines, implementing lakehouse architecture, optimizing Spark workloads, and integrating cloud data platforms. The candidate will work with data architects, analysts, and engineering teams to deliver scalable and reliable data solutions., * Design and develop scalable data engineering solutions using Databricks.
- Build batch and streaming data pipelines using Apache Spark.
- Develop data transformations using PySpark and Spark SQL.
- Implement lakehouse architecture using Delta Lake.
- Manage data access, governance, and permissions through Unity Catalog.
- Build and maintain automated workflows using Databricks Workflows.
- Implement incremental data processing and CDC pipelines.
- Optimize Spark jobs and improve processing performance.
- Integrate Databricks with cloud storage and enterprise databases.
- Develop data validation and quality monitoring processes.
- Collaborate with data scientists and analytics teams.
- Troubleshoot production issues and ensure pipeline reliability.
Requirements
- Strong hands-on experience with Databricks.
- Advanced programming skills in Python and PySpark.
- Strong knowledge of Apache Spark and Spark SQL.
- Experience with Delta Lake and lakehouse architecture.
- Knowledge of Unity Catalog and data governance.
- Experience with Delta Live Tables / Lakeflow.
- Experience building ETL and ELT pipelines.
- Knowledge of AWS, Azure, or GCP cloud platforms.
- Experience with SQL, data modeling, and data warehousing.
- Familiarity with Git, CI/CD, and workflow orchestration., * Experience with Databricks Lakeflow and Lakeflow Connect.
- Knowledge of Apache Kafka and real-time data processing.
- Experience with Terraform and cloud infrastructure automation.
- Knowledge of MLflow and machine learning workflows.
- Experience with enterprise data migration projects.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦