> Markdown version of [/jobs/ext/1075252-databricks-lead](https://www.wearedevelopers.com/jobs/ext/1075252-databricks-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Databricks Lead

- **Company:** Tata Consultancy Services Limited
- **Location:** Raleigh, NC, United States
- **Experience:** Expert
- **Salary:** $100,000.0 - $140,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Business Analytics Applications, Data Analysis, Microsoft Azure, Information Engineering, Extract Transform Load (ETL), Data Transformation, Python (Programming Language), Performance Tuning, Power BI, DataOps, Search Technologies, SQL Databases, Unstructured Data, Cloud Platform System, Large Language Models, Prompt Engineering, Apache Spark, Pyspark, Information Technology, Data Pipelines, Databricks
- **Published:** June 1, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=18312310b2db3bfe

## About the Role

Do you have experience in System tuning?, Do you have a Bachelor's degree?, * 10+ years overall experience in data engineering or related fields.

 * 3-5 years hands-on experience with Databricks and Spark.

 * Strong proficiency in SQL and data analysis techniques.

 * Experience with ETL processes, data modeling, and performance tuning.

 * Familiarity with Python or Scala for data engineering tasks.

 * Excellent problem-solving and communication skills., Qualifications : BACHELOR OF COMPUTER SCIENCE

## Description

We are seeking a hands-on Databricks Data Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong experience in data engineering, ETL development, and production support, ensuring reliable, scalable, and high-performing data operations within Azure environment and can work in a fast-paced environment. Knowledge of insurance domain and Power BI is a plus but not mandatory.
Development

 * Design, develop, and deploy scalable ETL/ELT data pipelines using Apache Spark, PySpark, and Databricks.

 * Develop and optimize SQL queries for data transformation and analysis.

 * Collaborate with product owners, data architects and analysts to build data models, delta lake structures, and data workflows.

 * Collaborate with data analysts and business teams to deliver actionable insights.

 * Build job orchestration and monitoring solutions

 * Ensure data quality, performance, and reliability across workflows.

 * Develop and maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows.

 * Work with cloud-based data platforms (Azure preferred).

GenAI & Databricks Genie Development

 * Build natural language query interfaces (Genie / AI assistants) over enterprise data

 * Develop RAG-based solutions integrating structured & unstructured data

 * Implement prompt engineering, embeddings, vector search, and LLM orchestration


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [REST, GraphQL, gRPC, and more: A comparison of modern API styles](https://www.wearedevelopers.com/videos/100247-rest-graphql-grpc-and-more-a-comparison-of-modern-api-styles)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)