Data Engineer
Proteam Solutions
Dublin, OH, United States
about 1 month ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Amazon Web Services
Application Frameworks
Microsoft Azure
Computer Programming
Continuous Integration
Information Engineering
Extract Transform Load (ETL)
Data Warehousing
Distributed Data Store
Apache Hive
Performance Tuning
+11 more
Data Streaming
Delivery Pipeline
Apache Spark
Git
Data Layers
Build Management
Data Lakes
Pyspark
Integration Frameworks
Data Pipelines
Databricks
Job description
We are seeking an experienced Senior Data Engineer with deep expertise in Databricks to design, develop, and optimize enterprise-scale data pipelines. You’ll play a key role in building scalable Lakehouse solutions while ensuring reliable, high-quality data for analytics, reporting, and downstream applications., * Design and build scalable ETL/ELT pipelines using Databricks
- Develop PySpark and Spark SQL transformations
- Build Bronze, Silver, and Gold data layers using Medallion Architecture
- Implement Delta Live Tables, Workflows, and Unity Catalog
- Optimize Spark jobs for scalability and performance
- Design dimensional data models for analytical workloads
- Implement data quality validations and monitoring
- Collaborate with analytics, engineering, and architecture teams
- Build reusable frameworks and automation
- Support CI/CD deployment pipelines and Git-based development
Requirements
- 7-10+ years of Data Engineering experience
- Extensive hands-on Databricks experience
- Strong PySpark programming
- Advanced Spark SQL skills
- Experience with Delta Lake
- Experience implementing Medallion Architecture
- Hands-on Delta Live Tables
- Unity Catalog experience
- Strong SQL optimization and performance tuning
- Experience with AWS, Azure, or GCP
- CI/CD and Git experience
- Strong understanding of distributed data systems
- Experience designing enterprise data models, * Databricks certification
- Experience with streaming data
- Experience building reusable data frameworks
- Experience with enterprise governance
- Exposure to AI-assisted development tools including Databricks AI Genie
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.indeed.comGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
BB
Benedikt Bischof
about 4 years ago
DS
Dhannush Subramani
Top Big Data Technologies That You Need to Know
about 4 years ago
EM
Eli McGarvie
Data Engineer Salary UK
about 3 years ago
CH
Chris Heilmann
Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production
almost 2 years ago
EM
Eli McGarvie
Highest Paying Tech Companies for Developers
over 3 years ago
LM
Luis Minvielle
How to Become an AI Engineer
over 2 years ago