Solutions Architect - Databricks

EXL SERVICE
United States
3 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
6 years minimum
Working hours
Regular working hours
Job source

Tech stack

Java (Programming Language) Airflow Big Data Cloud Engineering Code Review Data Architecture Information Engineering Data Infrastructure Extract Transform Load (ETL) Data Mart Data Warehousing Dimensional Modeling
+16 more
Distributed Data Store Distributed Systems Python (Programming Language) Online Analytical Processing Online Transaction Processing Performance Tuning Query Optimization Software Deployment SQL Databases Technical Data Management Systems Real Time Systems Apache Spark Pyspark Apache Kafka Data Pipelines Databricks

Job description

Job Description: We are seeking a highly skilled and hands-on Data Architect / Lead Data Engineer to design, build, and scale modern data platforms leveraging the Databricks Lakehouse architecture. This role combines deep technical expertise, architecture design, and client-facing leadership, with an emphasis on driving Databricks adoption across enterprise data ecosystems., * Partner with client stakeholders to define data platform strategy and establish Databricks Lakehouse architecture as the standard.

  • Design scalable, secure, and high-performance data architectures across batch and real-time processing.
  • Build and present reference architectures, solution blueprints, and demos to drive adoption and technical buy-in.
  • Translate complex business requirements into robust technical data solutions.
  • Design and implement scalable data pipelines using Databricks (PySpark, SQL).
  • Build and optimize data models, data marts, and medallion architecture layers (Bronze/Silver/Gold).
  • Develop and manage ETL/ELT pipelines, including CDC, incremental processing, and performance tuning.
  • Ensure data quality, observability, monitoring, and alerting across production workloads.
  • Work with large-scale distributed data systems (Spark, Kafka, etc.).
  • Lead and mentor a team of data engineers across multiple workstreams.
  • Conduct code reviews, enforce best practices, and create reusable frameworks/patterns.
  • Drive end-to-end solution delivery, including architecture, development, and production deployment.
  • Collaborate with cross-functional teams including analytics, BI, data science, and cloud engineering.
  • Manage stakeholder communication and provide technical thought leadership.

Expected work split:

  • 50% Hands-on Technical: Data engineering, pipeline development, architecture implementation
  • 50% Leadership & Client Engagement: Solution design, stakeholder management, team leadership

Requirements

Do you have experience in Team development?, * 6-10 years of experience in data engineering, data architecture, or analytics engineering

  • Strong experience with Databricks ecosystem (Spark, SQL, PySpark, workflows)
  • Expertise in:
  • Big Data Technologies (Spark, Kafka, distributed systems)
  • Data Warehousing & Modeling (OLTP/OLAP, dimensional modeling, medallion architecture)
  • ETL/ELT pipeline development and orchestration (Airflow or similar tools)
  • Advanced proficiency in SQL and Python (Scala/Java/R is a plus)
  • Experience designing and delivering enterprise-grade data architectures
  • Strong understanding of performance tuning, query optimization, and large-scale data processing
  • Proven ability to translate business needs into scalable technical solutions
  • Experience leading teams and working in client-facing environments
  • Excellent communication, problem-solving, and stakeholder management skills

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

2:34 min

Capabilities of the Apache Spark processing engine

Ayon Roy · LIVE

Videos

See all

Related articles

See all