Data Engineer

Proteam Solutions
Dublin, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Dublin, United States of America

Tech stack

Artificial Intelligence
Amazon Web Services (AWS)
Application Frameworks
Azure
Computer Programming
Continuous Integration
Information Engineering
ETL
Data Warehousing
Distributed Data Store
Hive
Performance Tuning
Data Streaming
Delivery Pipeline
Spark
GIT
Data Layers
Build Management
Data Lake
PySpark
Integration Frameworks
Data Pipelines
Databricks

Job description

We are seeking an experienced Senior Data Engineer with deep expertise in Databricks to design, develop, and optimize enterprise-scale data pipelines. You'll play a key role in building scalable Lakehouse solutions while ensuring reliable, high-quality data for analytics, reporting, and downstream applications., * Design and build scalable ETL/ELT pipelines using Databricks

  • Develop PySpark and Spark SQL transformations
  • Build Bronze, Silver, and Gold data layers using Medallion Architecture
  • Implement Delta Live Tables, Workflows, and Unity Catalog
  • Optimize Spark jobs for scalability and performance
  • Design dimensional data models for analytical workloads
  • Implement data quality validations and monitoring
  • Collaborate with analytics, engineering, and architecture teams
  • Build reusable frameworks and automation
  • Support CI/CD deployment pipelines and Git-based development

Requirements

  • 7-10+ years of Data Engineering experience
  • Extensive hands-on Databricks experience
  • Strong PySpark programming
  • Advanced Spark SQL skills
  • Experience with Delta Lake
  • Experience implementing Medallion Architecture
  • Hands-on Delta Live Tables
  • Unity Catalog experience
  • Strong SQL optimization and performance tuning
  • Experience with AWS, Azure, or GCP
  • CI/CD and Git experience
  • Strong understanding of distributed data systems
  • Experience designing enterprise data models, * Databricks certification
  • Experience with streaming data
  • Experience building reusable data frameworks
  • Experience with enterprise governance
  • Exposure to AI-assisted development tools including Databricks AI Genie

Apply for this position