Databricks Resident Solution Architect

Role SummaryWe are seeking a senior Databricks Resident Solution Architect for a long-term remote contract engagement. In this embedded advisory role, you will act as the principal Databricks authority, leading end-to-end architecture, migration, and performance optimization for complex enterprise client environments. You will collaborate remotely with customer data engineering, data science, and C-suite leadership to design scalable Lakehouse architectures, set up unified data governance, tune Spark workloads, and accelerate enterprise AI/ML adoption. Key ResponsibilitiesEnterprise Lakehouse Architecture: Lead remote design and delivery of modern Databricks Lakehouse environments using Delta Lake, Unity Catalog, Delta Live Tables (DLT), and Databricks SQL. Embedded Technical Advisory: Serve as the client’s dedicated resident architect, translating business and analytics needs into technical implementations. Optimization & Performance Tuning: Conduct deep-dive audits on large-scale Apache Spark workloads, PySpark code, dynamic cluster allocations, and SQL queries to reduce runtime and cloud costs. Governance & Security: Architect robust access control, column/row-level security, data lineage, and privacy rules using Unity Catalog to meet domain-specific compliance requirements. Migration Strategy: Lead technical planning for migrating legacy data warehouses and big data platforms (e.g., Snowflake, Teradata, Hadoop, Redshift) over to Databricks. CI/CD & MLOps Infrastructure: Partner with engineering teams to establish automated CI/CD pipelines, MLflow tracking, and modern data operations practices. Candidate QualificationsCore Technical Requirements & CertificationsPriority Certification

Rural Contracts, Embedded Systems, Enterprise Applications, Performance Tuning/Optimization, Leadership, Customer/Client Research, Data Science, Artificial Intelligence (AI), Cloud Computing, SQL (Structured Query Language), Databricks Champion Certification (or Databricks Certified Solutions Architect Professional) is strongly preferred.Experience: 8 10+ years of total experience in data engineering/architecture, with 3 5+ years of dedicated, hands-on experience on Databricks. Stack Expertise: Deep knowledge of Apache Spark, Python/PySpark, SQL, Delta Lake, Unity Catalog, and MLflow. Cloud Platforms: Hands-on architecture experience on AWS, Azure, or GCP cloud ecosystems (networking, storage, security services). Remote Delivery: Proven track record of independently leading enterprise client architecture and communicating effectively with technical and business stakeholders in a 100% remote setting. Preferred Industry ContextStrong candidates will hold deep domain architecture experience in at least one of these verticals:1. Public SectorFamiliarity with federal, state, or municipal regulatory standard requirements (e.g., FedRAMP, CJIS, FISMA, HIPAA).Experience architecting secure, compliant cloud-data frameworks for public entities.2. Banking & Financial Services (BFSI)Hands-on design of high-throughput transactional analytics, risk models, or anti-fraud data pipelines.Strict adherence to SOC2, PCI-DSS, RBAC security patterns, data masking, and financial audit standards.3. Healthcare & Life SciencesExperience managing Sensitive Personal Data (PHI/PII) under HIPAA compliance utilizing Unity Catalog encryption and masking standards.Familiarity with healthcare schemas and data standards (e.g., FHIR, HL7, OMOP).