Databricks Engineer

Tata Consultancy Services Limited

Raleigh, NC, United States

10 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Compensation

$100,000.0 - $140,000.0

Working hours

Regular working hours

Job source

www.careerbuilder.com

Tech stack

Business Analytics Applications Data Analysis Microsoft Azure Cloud Computing Databases Continuous Delivery Continuous Integration Extract Transform Load (ETL) Data Transformation Python (Programming Language) Performance Tuning Systems Development Life Cycle

+11 more

Cloud Services DataOps SQL Databases Scripting Apache Spark Data Lakes Pyspark Data Management Data Pipelines Databricks Programming Languages

Job description

Databricks and Apache Spark
SQL and data analysis
ETL/ELT development and optimization
Data modeling and performance tuning
Python or Scala
Azure cloud-based data platforms
Production support and operational monitoring

Roles & Responsibilities

Design, develop, and deploy scalable ETL/ELT pipelines using Databricks, Spark, and PySpark.
Build and optimize SQL-based data transformation and analytics solutions.
Collaborate with product owners, data architects, analysts, and business teams on data models and workflows.
Develop Delta Lake structures and enterprise data pipelines.
Implement job orchestration, monitoring, and data quality controls.
Maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows.
Support Azure-based cloud data platforms and ensure reliable, high-performing data operations.

Requirements

Apache Spark, Architectural Analysis, Cloud Computing, Continuous Deployment/Delivery, Continuous Integration, Data Analysis, Data Management, Data Modeling, Data Quality, Database Extract Transform and Load (ETL), Microsoft Windows Azure, Operational Support, Performance Modeling, Performance Tuning/Optimization, Production Support, Python Programming/Scripting Language, Quality Monitoring, SQL (Structured Query Language), Scala Programming Language, Scalable System Development