Databricks Resident Solution Architect

Sysmind LLC
San Francisco, CA, United States
28 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Data Analysis Microsoft Azure Bioinformatics Continuous Integration Information Engineering Data Infrastructure Extract Transform Load (ETL) Data Systems DevOps Memory Management Github
+15 more
Machine Learning Performance Tuning Cloud Services Data Streaming Cloud Platform System Apache Spark Software Troubleshooting Data Lakes Gitlab-ci Data Analytics Data Management Machine Learning Operations Data Pipelines Jenkins Databricks

Job description

We are seeking a Databricks Resident Solution Architect with extensive experience in data engineering, data platforms, analytics, and modern data warehouse solutions. This client-facing role will lead Databricks implementations, provide architecture guidance, and work directly with clients to design and deliver scalable cloud-native data platforms and Databricks solutions., * Lead end-to-end Databricks implementations as a hands-on developer, technical lead, or solution architect.

  • Serve as a client-facing technical advisor, understanding business requirements and recommending appropriate Databricks solutions.
  • Design and implement cloud-native data platforms across AWS, Azure, and/or GCP.
  • Architect scalable ETL/ELT pipelines using Databricks, Delta Lake, Structured Streaming, and modern data engineering frameworks.
  • Design and optimize large-scale distributed data processing workloads for performance, scalability, and cost efficiency.
  • Apply advanced Apache Spark expertise, including partitioning, execution plans, memory management, and performance optimization.
  • Implement DevOps and CI/CD practices using Azure DevOps, GitHub Actions, GitLab CI/CD, Jenkins, or similar tools.
  • Apply MLOps principles for machine learning lifecycle management, model deployment, and monitoring.
  • Leverage Databricks Lakehouse capabilities including Delta Lake, Unity Catalog, Workflows, MLflow, and Delta Live Tables.
  • Troubleshoot complex data platform challenges and recommend architecture patterns aligned with business and technical requirements.
  • Support client discussions and demonstrate the value and capabilities of Databricks solutions.

Requirements

  • 12-15+ years of experience in data engineering, data platforms, data analytics, and modern data warehouse solutions.
  • 10+ years of consulting and client-facing delivery experience.
  • Demonstrated experience delivering 6-8+ end-to-end Databricks implementations.
  • Advanced Databricks certification, such as Databricks Data Engineering Professional certification, with completion of relevant learning paths and coursework.
  • Strong expertise designing and implementing cloud-native data platforms across AWS, Azure, and/or GCP, with deep hands-on expertise in at least one cloud platform.
  • Advanced knowledge of Apache Spark, including performance optimization, partitioning strategies, execution plans, memory management, and runtime internals.
  • Extensive hands-on experience with Databricks, Delta Lake, Structured Streaming, and scalable ETL/ELT pipelines.
  • Experience implementing production-grade DevOps and CI/CD for data solutions.
  • Working knowledge of MLOps, machine learning lifecycle management, model deployment, and monitoring.
  • Strong knowledge of the Databricks Lakehouse Platform and its core capabilities.
  • Experience tuning large-scale distributed workloads and designing highly performant, scalable, and cost-efficient solutions.
  • Strong troubleshooting and solution architecture skills.

Preferred Qualifications

  • Databricks Solutions Architect Champion program participation or equivalent advanced Databricks expertise.
  • Experience working with public sector organizations

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law. We are committed to providing reasonable accommodations to qualified applicants with disabilities and for sincerely held religious beliefs, as required by applicable law. Applicants who require a reasonable accommodation during the hiring process may request assistance. Employment may be contingent upon the successful completion of job-related pre-employment screening, which may include background and reference checks and, where permitted by applicable law and applicable to the position, drug testing. All such screening will be conducted in accordance with applicable federal, state, and local laws.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…