> Markdown version of [/jobs/ext/3827775-databricks-sme](https://www.wearedevelopers.com/jobs/ext/3827775-databricks-sme). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Databricks SME

- **Company:** Scicom Infrastructure Services
- **Location:** Atlanta, GA, United States
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Amazon S3, Data Analysis, Computing Platforms, Microsoft Azure, Cloud Engineering, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Github, Apache Hive, Identity and Access Management, JSON, Network Security, Machine Learning, Performance Tuning, Cloud Services, Runbook, SQL Databases, Data Streaming, Parquet, Google Cloud, Cloud Platform System, Data Classification, Snowflake, Apache Spark, Software Troubleshooting, Gitlab, Data Lakes, Pyspark, Avro, Data Management, Machine Learning Operations, Terraform, Stream Analytics, Data Pipelines, Key Vault, Databricks
- **Published:** October 18, 2026
- **Apply:** https://www.careerbuilder.com/job-details/databricks-sme-atlanta-ga--966a584c-a608-4c3e-9b83-8393df0d2211

## About the Role

7+ years in data engineering, analytics, or platform architecture 3-5+ years hands-on Databricks experience in production environments Proven experience applying the Databricks Well-Architected Framework Experience designing cloud-native lakehouse architectures Experience supporting mission-critical data platforms

Technical Skills

Databricks Lakehouse Platform Apache Spark (PySpark / Scala / Spark SQL) Delta Lake, Delta Live Tables, Structured Streaming Unity Catalog (governance, lineage, access controls) Cloud platforms: AWS, Azure, or GCP Infrastructure as Code (Terraform strongly preferred) CI/CD tools (GitHub Actions, Azure DevOps, GitLab, etc.) Data formats and protocols (Parquet, JSON, Avro)

Certifications Required:

Databricks Certified Data Engineer Professional Databricks Certified Professional Architect (or equivalent advanced certification)

Preferred / Additional Certifications

AWS Certified Solutions Architect (Associate or Professional) Azure Solutions Architect Expert Google Professional Data Engineer Databricks Machine Learning Professional Snowflake or other cloud data platform certifications

Soft Skills

Strong architectural decision-making and documentation skills Excellent communication with technical and non-technical stakeholders Ability to lead design reviews and architecture governance forums Strong troubleshooting and performance-tuning mindset

Nice-to-Have Experience

MLflow and MLOps architectures Real-time analytics and streaming pipelines Multi-region or cross-account data architectures Consulting or MSP delivery experience

## Description

Key Responsibilities:Architecture & Platform DesignDesign enterprise Databricks Lakehouse architectures aligned with the Databricks Well-Architected Framework Define reference architectures for batch, streaming, analytics, and ML workloads Select and standardize cluster, compute, and workspace architectures Design multi-workspace strategies (dev/test/prod, shared vs. isolated) Ensure architectures meet scalability, availability, and performance requirementsWell-Architected Framework Alignment Apply Databricks best practices across all pillars, including:Security & Governance (Unity Catalog, IAM, data access controls) Reliability & Resilience (job retries, checkpointing, failure isolation) Performance Efficiency (cluster sizing, autoscaling, caching) Cost Optimization (compute policies, workload separation, monitoring) Operational Excellence (monitoring, automation, CI/CD, runbooks)Implementation & EngineeringLead Databricks workspace, cluster, and Unity Catalog implementations Implement Delta Lake, Delta Live Tables (DLT), and Structured Streaming Build and optimize ETL/ELT pipelines using Spark and SQL Integrate Databricks with cloud services (S3/ADLS/GCS, IAM, Key Vault, networking) Establish CI/CD pipelines for notebooks, jobs, and infrastructureSecurity, Governance & ComplianceImplement Unity Catalog for centralized governance Define data classification, lineage, and audit strategies Enforce least-privilege access and secure networking patterns Support compliance requirements (HIPAA, SOC 2, PCI, GDPR as applicable)Operations & OptimizationMonitor platform health, performance, and cost Troubleshoot production issues across jobs, clusters, and data pipelines Perform workload tuning and cost-performance optimization Define SLOs, alerts, and operational metricsCollaboration & LeadershipPartner with Data Engineering, Analytics, ML, Platform, and Security teams Translate business requirements into technical architectures Provide architectural guidance and technical mentorship Communicate risks, tradeoffs, and recommendations to leadershipRequired Qualifications:Experience7+ years in data engineering, analytics, or platform architecture 3-5+ years hands-on Databricks experience in production environments Proven experience applying the Databricks Well-Architected Framework Experience designing cloud-native lakehouse architectures Experience supporting mission-critical data platformsTechnical SkillsDatabricks Lakehouse Platform Apache Spark (PySpark / Scala / Spark SQL) Delta Lake, Delta Live Tables, Structured Streaming Unity Catalog (governance, lineage, access controls) Cloud platforms: AWS, Azure, or GCP Infrastructure as Code (Terraform strongly preferred) CI/CD tools (GitHub Actions, Azure DevOps, GitLab, etc.) Data formats and protocols (Parquet, JSON, Avro)Certifications Required:Databricks Certified Data Engineer Professional Databricks Certified Professional Architect (or equivalent advanced certification)Preferred / Additional CertificationsAWS Certified Solutions Architect (Associate or Professional) Azure Solutions Architect Expert Google Professional Data Engineer Databricks Machine Learning Professional Snowflake or other cloud data platform certificationsSoft SkillsStrong architectural decision-making and documentation skills Excellent communication with technical and non-technical stakeholders Ability to lead design reviews and architecture governance forums Strong troubleshooting and performance-tuning mindsetNice-to-Have ExperienceMLflow and MLOps architectures Real-time analytics and streaming pipelines Multi-region or cross-account data architectures Consulting or MSP delivery experience