Databricks & Data Engineering

Role Descriptions: Design| develop| and maintain scalable| high-performance data engineering solutions on Azure Databricks.Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.Design and implement Lakehouse and Medallion Architecture (Bronze| Silver| Gold) using Delta Lake.Develop reusable ETL/ELT frameworks| standardized data processing components| and enterprise data ingestion patterns.Ingest and process large-scale data from diverse enterprise sources| including databases| APIs| message queues| and file-based systems.Develop complex data transformation logic using PySpark| Python| and SQL.Create trusted| governed| and consumption-ready datasets for reporting| analytics| and downstream applications.Implement data quality| validation| reconciliation| audit| and governance controls across data pipelines.Manage and optimize Databricks Workflows| Delta Live Tables (DLT)| Unity Catalog| and Databricks SQL.Collaborate with business| analytics| architecture| and delivery teams in an Agile environment to deliver high-quality data products. Required Technical Skills:Databricks & Data EngineeringStrong hands-on experience with Databricks| Lakehouse Architecture| and Delta LakeExpertise in PySpark| Python| and advanced SQLExperience with Apache Spark| Structured Streaming| Delta Live Tables (DLT)| Databricks Workflows| and Unity CatalogExperience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutionsStrong background in data ingestion| ETL/ELT| data transformation| optimization| and deployment practicesExperience with event-driven and streaming technologies such as Kafka or Azure Event HubCloud PlatformsMicrosoft Azure (preferred)AWS or GCP experience is also valuableApplication IntegrationWorking knowledge of React and/or Angular for front-end integration and data-centric application developmentDomain Experience (Preferred)Experience in Finance| Insurance| Regulatory Reporting| or US GAAP reportingUnderstanding of financial data models| reporting processes| and governance requirements is highly desirable Essential Skills: Design| develop| and maintain scalable| high-performance data engineering solutions on Azure Databricks.Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming.Design and implement Lakehouse and Medallion Architecture (Bronze| Silver| Gold) using Delta Lake.Develop reusable ETL/ELT frameworks| standardized data processing components| and enterprise data ingestion patterns.Ingest and process large-scale data from diverse enterprise sources| including databases| APIs| message queues| and file-based systems.Develop complex data transformation logic using PySpark| Python| and SQL.Create trusted| governed| and consumption-ready datasets for reporting| analytics| and downstream applications.Implement data quality| validation| reconciliation| audit| and governance controls across data pipelines.Manage and optimize Databricks Workflows| Delta Live Tables (DLT)| Unity Catalog| and Databricks SQL.Collaborate with business| analytics| architecture| and delivery teams in an Agile environment to deliver high-quality data products. Required Technical Skills:Databricks & Data EngineeringStrong hands-on experience with Databricks| Lakehouse Architecture| and Delta LakeExpertise in PySpark| Python| and advanced SQLExperience with Apache Spark| Structured Streaming| Delta Live Tables (DLT)| Databricks Workflows| and Unity CatalogExperience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutionsStrong background in data ingestion| ETL/ELT| data transformation| optimization| and deployment practicesExperience with event-driven and streaming technologies such as Kafka or Azure Event HubCloud PlatformsMicrosoft Azure (preferred)AWS or GCP experience is also valuableApplication IntegrationWorking knowledge of React and/or Angular for front-end integration and data-centric appl

Responsibilities “ Design, develop, and maintain scalable, high-performance data engineering solutions on Azure Databricks. “ Build and support batch and real-time data pipelines using Apache Spark and Structured Streaming. “ Design and implement Lakehouse and Medallion Architecture (Bronze, Silver, Gold) using Delta Lake. “ Develop reusable ETL/ELT frameworks, standardized data processing components, and enterprise data ingestion patterns. “ Ingest and process large-scale data from diverse enterprise sources, including databases, APIs, message queues, and file-based systems. “ Develop complex data transformation logic using PySpark, Python, and SQL. “ Create trusted, governed, and consumption-ready datasets for reporting, analytics, and downstream applications. “ Implement data quality, validation, reconciliation, audit, and governance controls across data pipelines. “ Manage and optimize Databricks Workflows, Delta Live Tables (DLT), Unity Catalog, and Databricks SQL. “ Collaborate with business, analytics, architecture, and delivery teams in an Agile environment to deliver high-quality data products.

Cisco Unity, React.js, Insurance, GCP (Good Clinical Practices), Application Programming Interface (API), Apache Spark, Streaming Technology, Python Programming/Scripting Language, Finance, Data Management, Databricks & Data Engineering “ Strong hands-on experience with Databricks, Lakehouse Architecture, and Delta Lake “ Expertise in PySpark, Python, and advanced SQL “ Experience with Apache Spark, Structured Streaming, Delta Live Tables (DLT), Databricks Workflows, and Unity Catalog “ Experience designing and implementing Medallion Architecture and modern Data Lake/Data Warehouse solutions “ Strong background in data ingestion, ETL/ELT, data transformation, optimization, and deployment practices “ Experience with event-driven and streaming technologies such as Kafka or Azure Event Hub Cloud Platforms “ Microsoft Azure (preferred) “ AWS or GCP experience is also valuable Application Integration “ Working knowledge of React and/or Angular for front-end integration and data-centric application development Domain Experience (Preferred) “ Experience in Finance, Insurance, Regulatory Reporting, or US GAAP reporting “ Understanding of financial data models, reporting processes, and governance requirements is highly desirable