Senior Databricks Data Engineer

Tata Consultancy Services Limited
New York, NY, United States
1 day ago
Apply on jobs.localjobnetwork.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$110,000.0 - $120,000.0
Working hours
Regular working hours

Tech stack

Amazon S3 Continuous Integration Data Validation Extract Transform Load (ETL) Apache Hive Performance Tuning Data Streaming Workflow Management Systems Git Build Management Data Lakes Pyspark
+4 more
Information Technology Deployment Automation Data Pipelines Databricks

Job description

Must Have Technical/Functional Skills Databricks, PySpark, Spark SQL and Delta Lake Auto Loader, Structured Streaming, AWS S3 and ETL/ELT Workflow orchestration, Git/CI/CD and performance tuning Data validation, reconciliation and production support Roles & Responsibilities Design and build production Databricks ingestion and transformation pipelines Implement batch/streaming processing and Delta Lake optimization Automate deployment and scheduling through CI/CD and workflows Monitor, reconcile and troubleshoot data pipelines In addition, should be able to do the following

  1. Play the role of onsite co-Ordinator for offshore
  2. Able to conduct RCA and provide and Implement solutions

Requirements

Customer-facing communication and concise executive reporting Onsite-offshore coordination across distributed teams Structured ownership, prioritization, risk escalation and collaboration Ability to work independently while following enterprise governance and security standards, BACHELOR OF COMPUTER SCIENCE

Benefits & conditions

Salary Range: $110,000-$120,000 a year TCS Employee Benefits Summary: Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement. Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on jobs.localjobnetwork.com
Prepare application

Good distractions

Loading talks and stories from around this role…