Databricks Data Engineer

Voltijds

13 uren geleden

Help design, build and continuously improve the clients online platform.

Research, suggest and implement new technology solutions following best practices/standards.

5+ years of Data Engineering experience.

3+ years of hands-on Databricks experience.

Play a key role in designing, building, and optimizing scalable data solutions on Databricks (AWS), enabling trusted data products, data governance, and AI-driven business capabilities.

Strong data engineering expertise with hands-on experience in Databricks, cloud-based data platforms, data modeling, and modern software engineering practices.

Hands-on Databricks Data Engineer who can independently design and implement data solutions, collaborate effectively with business and technical stakeholders, and contribute to delivering trusted, governed, and scalable data products.

Develop and optimize ETL/ELT processes for large-scale enterprise data workloads.

Implement data models and data products following Data Mesh and Medallion Architecture principles.

Work with structured, semi-structured, and streaming data sources.

Develop solutions using PySpark, SQL, and Databricks Workflows.

Integrate and govern data using Unity Catalog.

Support data quality, metadata management, lineage, and governance initiatives.

Optimize data processing for performance, scalability, reliability, and cost efficiency.

Collaborate with Data Architects, Product Owners, Data Governance teams, and Business Stakeholders.

Contribute to CI/CD, automated testing, and deployment processes.

Strong Python and PySpark development skills.

Advanced SQL knowledge with experience in performance optimization.

Experience with Delta Lake and Medallion Architecture.

Experience with Databricks Unity Catalog.

Experience building scalable ETL/ELT pipelines.

Strong understanding of data modeling and data warehousing concepts.

Experience with Git, CI/CD, and DevOps practices.

Experience working in Agile delivery teams.

Experience with Data Governance platforms and metadata management.

Knowledge of Data Quality frameworks and monitoring solutions.

Experience with Data Products and Data Mesh concepts.

Experience with Databricks Workflows and Asset Bundles.

Knowledge of AI, Machine Learning, or GenAI solutions.

Experience working in regulated enterprise environments.

Preferred Technology Stack: Databricks, PySpark, Python, SQL, Delta Lake, Unity Catalog, Git/GitLab, CI/CD, Data Quality Frameworks, Data Governance Tooling, REST APIs, Cloud Data Platforms.