IT Resident Solutions Architect

Resident Solutions Architect - Databricks, Location: 100% Remote - USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory, We are seeking a hands-on, customer-facing Resident Solutions Architect with deep Databricks Platform expertise, combined with strong CI/CD and Terraform skills. This is an architect-level role for someone who can work directly with enterprise clients, design scalable Lakehouse and data platform solutions, and stay hands-on through build, deployment, and production support - including infrastructure-as-code delivery of the Databricks platform itself., · Serve as a resident, customer-facing Databricks Solutions Architect, providing architectural guidance, best practices, and hands-on delivery for enterprise Lakehouse implementations.

· Design and implement scalable Lakehouse architectures using PySpark, Spark SQL, and Delta Lake.

· Own Unity Catalog governance and security design, including data access controls, lineage, and cross-workspace governance patterns.

· Build and operate production data pipelines using Delta Live Tables (DLT) and Databricks Workflows.

· Administer Databricks Workspaces and Accounts, including provisioning, access management, and platform configuration.

· Provision and manage Databricks infrastructure using Terraform, including workspace, cluster, Unity Catalog, and job/workflow resources as reusable, version-controlled modules.

· Design and implement CI/CD pipelines for Databricks assets (notebooks, DLT pipelines, jobs, ML models) using GitHub Actions, GitLab CI, Jenkins, Azure DevOps, or equivalent, including Databricks Asset Bundles.

· Perform Spark performance tuning - partitioning, caching, Adaptive Query Execution (AQE), and data skew mitigation - to optimize cost and performance at scale.

· Design and manage SQL Warehouses for BI and analytics workloads.

· Build streaming data solutions using Kafka and Structured Streaming.

· Architect Databricks solutions across AWS, Azure, or Google Cloud Platform, tailored to each cloud’s native services and security model.

· Partner directly with client engineers, business stakeholders, and executives to translate business problems into Databricks/Lakehouse solutions and measurable outcomes.

· Guide client teams through migration from legacy ETL platforms (e.g., Informatica) to Databricks.

USA Experience: 15+ Years overall (5+ Years hands-on Databricks Platform experience required) Databricks Certification: Mandatory, · 15+ years of experience in Data Engineering, Cloud Engineering, Solutions Architecture, or Platform Engineering.

· Databricks Certification required (e.g., Databricks Certified Data Engineer Professional, Databricks Certified Solutions Architect, or equivalent).

· 5+ years of hands-on Databricks Platform experience, including:

· PySpark, Spark SQL, and Delta Lake

· Lakehouse Architecture design and implementation

· Unity Catalog governance and security

· Delta Live Tables (DLT)

· Databricks Workflows

· Workspace and Account Administration

· SQL Warehouses

· Strong hands-on Terraform experience with Databricks, including reusable modules for workspace, cluster, and Unity Catalog provisioning.

· Strong CI/CD experience (GitHub Actions, GitLab CI/CD, Jenkins, Azure DevOps, or equivalent), including CI/CD for Databricks Asset Bundles, notebooks, and jobs.

· Strong Spark performance tuning expertise - partitioning, caching, AQE, and data skew resolution.

· Kafka and Structured Streaming experience.

· Strong experience with AWS, Azure, or Google Cloud Platform.

· Proven customer-facing experience providing architectural guidance and Databricks best practices to enterprise clients.

· Strong communication and stakeholder-management skills, with the ability to translate business needs into practical Lakehouse solutions.

Preferred Skills

· MLflow for model tracking and lifecycle management.

· Databricks AI capabilities (AI/BI, Databricks Assistant, Mosaic AI).

· Experience migrating legacy ETL platforms (e.g., Informatica) to Databricks.

· Familiarity with the latest Databricks features - Genie, Lakebase, and Databricks Apps.

· Experience working with enterprise clients in a consulting or professional-services environment.

· Kubernetes, Docker, and cloud-native application architecture.