Databricks Developer

Tata Consultancy Services Limited

Raleigh, NC, United States

about 2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Compensation

$100,000.0 - $140,000.0

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Unity 3d Artificial Intelligence Amazon Web Services Microsoft Azure Cloud Computing Information Engineering Extract Transform Load (ETL) Python (Programming Language) Machine Learning Power BI SQL Databases Tableau (Software)

+6 more

Data Processing Large Language Models Data Lakes Virtual Agents Data Pipelines Databricks

Job description

Design and develop scalable data pipelines and ETL workflows using Databricks (Spark, PySpark)
Build and optimize Delta Lake architectures for high-performance data processing
Develop and deploy AI/ML models and advanced analytics solutions
Implement Agentic AI / GenAI solutions using Databricks (e.g., Agent Bricks, LLM-based workflows)
Develop BI and analytics datasets for downstream reporting and visualization
Collaborate with business and data science teams to translate requirements into solutions
Optimize performance, cost, and scalability of data pipelines on cloud platforms
Ensure data quality, governance, and compliance standards are met

Requirements

Design and develop scalable data pipelines and ETL workflows using Databricks (Spark, PySpark)
Build and optimize Delta Lake architectures for high-performance data processing
Develop and deploy AI/ML models and advanced analytics solutions
Implement Agentic AI / GenAI solutions using Databricks (e.g., Agent Bricks, LLM-based workflows)
Develop BI and analytics datasets for downstream reporting and visualization
Collaborate with business and data science teams to translate requirements into solutions
Optimize performance, cost, and scalability of data pipelines on cloud platforms
Ensure data quality, governance, and compliance standards are met
Excellent problem-solving and communication skills., * Strong hands-on experience with Databricks (Workflows, Delta Lake, Unity Catalog)
Expertise in ETL/Data Engineering using Spark/PySpark
Experience with AI/ML model development and deployment
Knowledge of Agentic AI / GenAI frameworks and LLM integration
Strong understanding of cloud platforms (AWS / Azure / GCP)
Experience with BI tools (Power BI, Tableau, etc.)
Proficiency in SQL, Python, Qualifications : BACHELOR OF COMPUTER SCIENCE