Data Engineer
Bright Sol
Alpharetta, GA, United States
about 2 months ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
3 years minimum
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Airflow
Apache HTTP Server
Microsoft Azure
Continuous Integration
Information Engineering
Extract Transform Load (ETL)
Data Systems
Data Warehousing
Python (Programming Language)
Metadata
Standard Sql
+15 more
Azure Data Lake
Data Streaming
Workflow Management Systems
Snowflake
Apache Spark
Git
Data Lakes
Pyspark
Star Schema
Apache Kafka
Spark Streaming
Data Management
Azure Synapse Analytics
Data Pipelines
Databricks
Job description
- Design and develop scalable data pipelines using Palantir Foundry, Databricks, and Snowflake
- Build ETL/ELT workflows using Python, PySpark, and SQL
- Develop Azure-based data solutions using ADF, ADLS, Synapse, and Databricks
- Support enterprise data migration and modernization initiatives
- Implement data quality, governance, metadata, and lineage
- Optimize Spark/Databricks workloads and Snowflake queries
- Develop dimensional data models and analytical datasets
- Support batch and near-real-time data pipelines
- Troubleshoot production pipeline, data quality, and performance issues
- Implement CI/CD and collaborate with data scientists, analysts, and business stakeholders
Preferred / Nice-to-Have
- Delta Lake / Apache Iceberg
- Unity Catalog
- Kafka / Spark Streaming / Azure Event Hubs
- Snowflake clustering and materialized views
- IBM WatsonX.data or similar AI data platforms
- GenAI / RAG / LangChain
- MCP
Requirements
- 3+ years of hands-on Data Engineering experience
- Strong Python, PySpark, and SQL
- Hands-on Databricks experience
- Strong Snowflake experience
- Azure experience with ADF, ADLS Gen2, Synapse, and Databricks
- Hands-on Palantir Foundry experience
- ETL/ELT pipeline development
- Data modeling - Star Schema / Snowflake Schema
- Airflow, dbt, or similar orchestration tools
- Data quality, governance, metadata, and security
- Git and CI/CD
- Batch and/or streaming data pipelines
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦