Databricks Solution Architect
Role details
Job location
Tech stack
Job description
We are looking for a highly skilled and passionate Databricks Solution Architect to design end-to-end data solutions, define architecture patterns, guide implementation teams, ensure integration with enterprise systems, and promote best practices in data engineering and machine learning workflows. Lead proof-of-concept developments and evaluate new platform features, and ensure that security and compliance standards are met Responsibilities:
-
Work closely with Data Scientists, Business Team and Engineers to understand business requirements, translating those requirements into technical architectures, and ensuring the effective integration of Databricks with other enterprise systems.
-
Recommend best practices around data engineering, machine learning, and analytics workflows, as well as guiding teams in the adoption of efficient architecture patterns.
-
Define project scope, set technical direction, and troubleshoot complex problems.
-
Lead the development of proof-of-concept solutions, evaluate new platform features, and ensure that security and compliance standards are met.
-
Plan and execute Migration strategies to transition on prem data warehouse and Datalake into cloud based Databricks
Requirements
Do you have experience in System tuning?, Do you have a Bachelor's degree?, * Proven expertise in implementing Lakehouse and Delta Lake using Databricks.
-
Strong PySpark and Python experience
-
Databricks (AWS) Platform Architect Certification
-
Familiarity with ML Ops/LLM Ops and distributed systems.
-
Experience with Big Data platform like Cloudera Hadoop and Could platforms like AWS, GCP.
-
Solid understanding of system design patterns, scalability, observability, and performance tuning.
-
Strong analytical and problem-solving skills.
-
Passion for exploring and building with emerging technologies.
Good to Have Skills:
- AWS EKS Experience, Dockers and Containers, Qualifications : BACHELOR OF COMPUTER SCIENCE