> Markdown version of [/jobs/ext/3403996-databricks-sme](https://www.wearedevelopers.com/jobs/ext/3403996-databricks-sme). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Databricks SME

- **Company:** CSCI Consulting
- **Location:** Indianapolis, IN, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Amazon Web Services, Data Analysis, Microsoft Azure, Big Data, Cloud Engineering, Cyber Security, Databases, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Data Integration, Extract Transform Load (ETL), Data Security, Data Transformation Services, Data Warehousing, Distributed Computing Environment, Data Flow Control, Github, Python (Programming Language), Machine Learning, Meta-Data Management, Oracle (Applications), Performance Tuning, Power BI, Cloud Services, Salesforce.Com, SAP (Applications), SQL Databases, Systems Integration, Tableau (Software), Enterprise Data Management, Enterprise Software Applications, Cloud Platform System, Apache Spark, Multi-Cloud, Generative AI, Gitlab, Data Layers, Microsoft Fabric, Containerization, Data Lakes, Pyspark, Kubernetes, Infrastructure Automation Frameworks, Information Technology, Qlikview, Bicep, Data Management, Machine Learning Operations, Terraform, Data Pipelines, Devsecops, Jenkins, Servicenow, Databricks
- **Published:** September 25, 2026
- **Apply:** https://www.clearancejobs.com/jobs/9196188/databricks-sme

## About the Role

* 7+ years of experience supporting enterprise data engineering, analytics, or data platform initiatives
 * 5+ years of hands-on experience with Databricks and Apache Spark
 * Experience designing and implementing enterprise data lake, data warehouse, or Lakehouse architecture
 * Strong expertise with Databricks Lakehouse Platform and Delta Lake
 * Advanced proficiency with SQL, Python, PySpark, and distributed data processing
 * Experience building and maintaining ETL/ELT pipelines and data integration solutions
 * Experience supporting Azure, AWS, or other cloud-based data platforms
 * Knowledge of data governance, data quality, metadata management, and data security best practices
 * Experience troubleshooting performance, scalability, and reliability issues in large-scale data environments
 * Experience developing technical documentation and communicating complex technical concepts to both technical and non-technical audiences
 * Ability to obtain and maintain the security clearance required by the program
 * U.S. Citizenship required where mandated by the applicable government contract
 * Able to obtain security clearance
 * Creativity and adaptability in problem-solving
 * Ability to work with clients to understand their needs
 * Strong organizational and time-management skills
 * Excellent written and verbal communication skills
 * Professional presence

Preferred Skills

 * Databricks Certified Data Engineer Associate, Professional Data Engineer, or related Databricks certification
 * Experience supporting Department of War, DoD, Intelligence Community, or Federal data modernization programs
 * Experience with Unity Catalog, Delta Live Tables, MLflow, Databricks Workflows, and Databricks Asset Bundles
 * Experience implementing modern data architecture including Data Mesh, Data Fabric, and Lakehouse strategies
 * Experience supporting Artificial Intelligence (AI), Machine Learning (ML), and Generative AI initiatives
 * Experience integrating Databricks with SAP, Salesforce, ServiceNow, Oracle, or other enterprise platforms
 * Experience with Power BI, Tableau, Qlik, or other enterprise reporting and visualization platforms
 * Experience implementing DevSecOps, CI/CD pipelines, and Infrastructure-as-Code frameworks
 * Experience using Azure DevOps, GitHub, GitLab, Jenkins, or other enterprise development platforms
 * Knowledge of Kubernetes, containerization technologies, and cloud-native architecture
 * Experience with Terraform, Bicep, or other infrastructure automation tools
 * Experience supporting data governance and compliance requirements within regulated environments
 * Experience working within Agile or SAFe delivery frameworks
 * Ability to work in a team environment, as well as independently
 * Strong customer and vendor relationship skills
 * Demonstrated ability to comply with data standards and policies
 * Motivation to learn new technologies and methodologies that demonstrate value
 * Past experience working with a Federal government agency
 * Department of War, etc. experience is a plus!

## Description

CSCI Consulting is looking for a Databricks Subject Matter Expert (SME) to support our Department of War client. In this role, the consultant will serve as the technical lead for Databricks-based data modernization initiatives, providing architecture, engineering, governance, analytics, and cloud data platform expertise to support mission-critical data, reporting, AI/ML, and decision-support capabilities., * Serve as the primary Databricks SME supporting enterprise data and analytics initiatives

 * Design, develop, and optimize scalable data platforms using the Databricks Lakehouse architecture
 * Lead the implementation of data engineering, data integration, and data transformation solutions using Databricks and Apache Spark
 * Design and implement Medallion Architecture frameworks, including bronze, silver, and gold data layers
 * Develop and maintain ETL/ELT pipelines for structured and unstructured data sources
 * Configure and optimize Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables
 * Support cloud-native data solutions within Azure, AWS, or multi-cloud environments
 * Collaborate with business stakeholders, architects, developers, and data teams to translate requirements into scalable solutions
 * Support data governance, metadata management, data quality, and master data management initiatives
 * Optimize platform performance, scalability, availability, and cost efficiency
 * Develop technical documentation, architecture diagrams, data-flow diagrams, and operational procedures
 * Support enterprise analytics, business intelligence, artificial intelligence, and machine learning initiatives
 * Implement and support DevSecOps, CI/CD, and Infrastructure-as-Code practices
 * Integrate Databricks solutions with enterprise applications, APIs, databases, and cloud services
 * Support cybersecurity, compliance, audit, and data protection requirements
 * Provide technical mentorship and guidance to engineers, analysts, and project teams
 * Participate in Agile ceremonies, technical reviews, solution demonstrations, and program planning activities


## Related Videos

- [Back(end) to the Future: Embracing the continuous Evolution of Infrastructure and Code](https://www.wearedevelopers.com/videos/440-back-end-to-the-future-embracing-the-continuous-evolution-of-infrastructure-and-code)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [GitLab CI pipelines for a whole company](https://www.wearedevelopers.com/videos/143-gitlab-ci-pipelines-for-a-whole-company)
- [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)