Databricks Solution Architect

Tata Consultancy Services Limited
Irving, United States of America
28 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Compensation
$ 140K

Job location

Irving, United States of America

Tech stack

Amazon Web Services (AWS)
Information Engineering
Data Infrastructure
Data Systems
Data Warehousing
Distributed Systems
Hadoop
Python
Machine Learning
Performance Tuning
Cloudera
Enterprise Software Applications
Large Language Models
Data Lake
PySpark
Information Technology
Machine Learning Operations
Amazon Web Services (AWS)
Databricks

Job description

We are looking for a highly skilled and passionate Databricks Solution Architect to design end-to-end data solutions, define architecture patterns, guide implementation teams, ensure integration with enterprise systems, and promote best practices in data engineering and machine learning workflows. Lead proof-of-concept developments and evaluate new platform features, and ensure that security and compliance standards are met Responsibilities:

  • Work closely with Data Scientists, Business Team and Engineers to understand business requirements, translating those requirements into technical architectures, and ensuring the effective integration of Databricks with other enterprise systems.

  • Recommend best practices around data engineering, machine learning, and analytics workflows, as well as guiding teams in the adoption of efficient architecture patterns.

  • Define project scope, set technical direction, and troubleshoot complex problems.

  • Lead the development of proof-of-concept solutions, evaluate new platform features, and ensure that security and compliance standards are met.

  • Plan and execute Migration strategies to transition on prem data warehouse and Datalake into cloud based Databricks

Requirements

Do you have experience in System tuning?, Do you have a Bachelor's degree?, * Proven expertise in implementing Lakehouse and Delta Lake using Databricks.

  • Strong PySpark and Python experience

  • Databricks (AWS) Platform Architect Certification

  • Familiarity with ML Ops/LLM Ops and distributed systems.

  • Experience with Big Data platform like Cloudera Hadoop and Could platforms like AWS, GCP.

  • Solid understanding of system design patterns, scalability, observability, and performance tuning.

  • Strong analytical and problem-solving skills.

  • Passion for exploring and building with emerging technologies.

Good to Have Skills:

  • AWS EKS Experience, Dockers and Containers, Qualifications : BACHELOR OF COMPUTER SCIENCE

Apply for this position