Databricks Solutions Architect 4

United Global Technologies

Los Alamos, NM, United States

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

7 years minimum

Working hours

Regular working hours

Job source

www.wayup.com

Tech stack

Artificial Intelligence Amazon S3 Cloud Engineering Cyber Security Continuous Integration Data Governance Extract Transform Load (ETL) Data Warehousing Electronic Data Interchange (EDI) Python (Programming Language) Key Management SAP ERP

+15 more

Cloud Services SAP (Applications) SQL Databases Data Streaming Data Classification Apache Spark Amazon Virtual Private Cloud (VPC) Data Lakes Spark Streaming Data Management Machine Learning Operations Oracle Ebusiness Terraform GXP Databricks

Job description

Advise on the Databricks reference architecture covering Lakeflow, Delta Lake, Unity Catalog Mosaic AI, Databricks SQL, and Databricks Apps
Provide hands-on guidance on Databricks full-stack solution design
Advise on design of both data movement (e.g. batch, real-time streaming) and zero-copy patterns
Guide platform migrations from legacy SQL and document stores onto Databricks and related cloud data platforms
Recommend security, governance, and compliance-as-code: Unity Catalog ABAC, governed tags, row filters/column masks, key management, networking security and compliance security profiles.
Advise on design of segregated/tiered data pool architectures aligned to the organization’s data classification policy and applicable regulatory requirements, including cross-tier data movement
Recommend FinOps and performance standards, workload and cost management
Mentor data engineers, analysts, and AI/ML teams participate in architecture reviews and contribute to standards
Work with information & cyber security, and other teams as needed for compliance to policies and standards
Evaluate emerging Databricks capabilities and recommend adoption strategies, including endpoint integrations with third parties as they evolve

Requirements

7+ years designing, building, and operating Databricks workloads with medallion layer architecture
Hands-on experience with Delta Lake, Unity Catalog,, Spark Structured Streaming, Lakeflow (or DLT/Live Tables), Databricks SQL, MLflow, and Mosaic AI Model Serving
Strong experience in cloud architecture supporting the platform (VPC/VNet, Security, S3/ADLS, KMS, etc.)
Proven experience migrating enterprise data workloads from legacy/on prem data warehouse/lake to Databricks
Production streaming and CDC experience
Depth in security and governance: Unity Catalog policies (ABAC), classification, encryption, access auditing, compliance-to-control translation
Hands-on experience designing multi-tier Databricks architectures driven by data classification requirements
Strong Python and SQL, Spark, Notebooks
Excellent verbal and written communication skills Nice to Have:
Active Databricks certifications
GenAI and agent systems on Databricks experience
Delta Sharing and Databricks Clean Rooms for controlled, governed data exchange across organizational/cross-tier/system boundaries
Familiarity with Oracle EBS and SAP enterprise data
Experience with SAP BDC/DataSphere and Databricks integration to SAP BDC, with zero-copy patterns
Lakeflow Designer, Databricks Apps, Genie, Databricks One
CI/CD, Infrastructure-as-Code (e.g. Terraform) for data platforms
Experience in designing, building, supporting BI reporting layers
Data Quality and Data Governance experience
Prior experience in regulated industries (e.g. utilities, life sciences, government services) and exposure to related frameworks (e.g. FERC/NERC, HIPAA, GxP, GDPR, FedRAMP, ITAR, etc.)