Databricks Engineer

· Data BI & Integration Platforms is seeking an experienced Cloud Platform Databricks SME, responsible for overseeing the Platform administration, Security, new NPI tools integration, migrations, platform maintenance and other platform administration activities on Azure/AWS., · Databricks Subject Matter Expert (SME) plays a pivotal role in admin, security best practices, platform sustain support, new tools adoption, cost optimization, supporting new patterns/design solutions using the Databricks platform. Here''s a breakdown of typical responsibilities:

· Core Technical Responsibilities

· Architect and optimize big data pipelines using Apache Spark, Delta Lake, and Databricks-native tools.

· Design scalable data ingestion and transformation workflows, including batch and streaming (e.g., Kafka, Spark Structured Streaming).

· Create integration guidelines to configure and integrate Databricks with other existing security tools relevant to data access control.

· Implement data security and governance using Unity Catalog, access controls, and data classification techniques.

· Support migration of legacy systems to Databricks on cloud platforms like Azure, AWS, or Google Cloud Platform.

· Manage cloud platform operations with a focus on FinOps support, optimizing resource utilization, cost visibility, and governance across multi-cloud environments.

· Act as a technical advisor to data engineering and analytics teams, guiding best practices and performance tuning.

· Partner with architects and business stakeholders to align Databricks solutions with enterprise goals.

Lead proof-of-concept (PoC) initiatives to demonstrate Databricks capabilities for specific use cases.

Must Have Skills:

Databricks Engineer

SME, Cloud, AZURE, Google Cloud Platform, AWS

· The ideal candidate will have hands-on experience with Azure/AWS services - Infrastructure as Code (IaC), platform provisioning & administration, cloud network design, cloud security principles and automation.