Databricks Platform Lead / Technical Delivery Lead

• Own the engineering and operational management of the Databricks platform supporting the enterprise pricing application. \n

• Lead end-to-end technical delivery of platform enhancements, environment changes, integrations, releases, and production improvements. \n

• Provision and administer Databricks accounts, workspaces, compute resources, clusters, policies, and supporting Azure services. \n

• Define and enforce platform standards covering architecture, security, governance, compute, deployment, monitoring, and cost management. \n

• Design and maintain appropriate separation across development, test, UAT, and production environments. \n

• Implement infrastructure as code and automated deployment practices for Databricks and supporting cloud resources. \n

• Establish secure connectivity between Databricks, SQL Server, cloud storage, the pricing engine, the underwriting workbench, and other enterprise systems. \n

• Configure and maintain identity, access, security, secrets, networking, and governance controls. \n

• Define Unity Catalog structures, access models, data ownership, lineage, and auditability standards. \n

• Establish cluster policies, runtime standards, compute controls, and workload-management practices. \n

• Monitor and optimize platform availability, reliability, performance, capacity, and cost. \n

• Troubleshoot workspace, authentication, networking, compute, storage, access, and platform-performance issues. \n

• Lead root-cause analysis and corrective actions for platform incidents and recurring operational problems. \n

• Provide technical oversight to data engineers building ingestion pipelines, transformations, curated datasets, and pricing data products. \n

• Review pipeline designs, Spark workloads, data-processing patterns, deployment approaches, and production-readiness plans. \n

• Ensure that data-engineering solutions comply with platform, security, governance, quality, and performance standards. \n

• Coordinate platform engineering, data engineering, application development, QA, DevOps, infrastructure, security, and production-support teams. \n

• Independently identify technical and delivery impacts across Databricks, the SQL Server pricing engine, the underwriting workbench, and integrated systems. \n

• Support successful SIT, UAT, regression testing, release validation, deployment, cutover, and post-production stabilization. \n

• Communicate technical risks, delivery dependencies, architectural decisions, and recovery plans to business and technology stakeholders. \n

• Serve as the primary technical point of contact for the Databricks platform and its integration with the enterprise pricing application. \n

• Serve as a subject matter expert on Databricks platform engineering, administration, governance, security, reliability, and operational best practices. \n

The ideal candidate must be capable of managing the Databricks platform foundation while providing technical oversight to data-engineering teams building ETL/ELT pipelines, transformations, and pricing data products.

\n

Technical and Delivery Expertise

\n \n

• 10+ years of overall technology experience. \n

• 5+ years of hands-on experience with Databricks, cloud data platforms, or platform engineering., * 3+ years of experience in a Platform Lead, Databricks Lead, Cloud Platform Engineer, Data Platform Lead, or similar role., * Prior experience implementing a Databricks-enabled pricing, underwriting, actuarial, insurance analytics, or financial-services application. \n

• Experience working with global insurance or reinsurance carriers., * Experience defining the division of responsibility among platform engineering, data engineering, application development, DevOps, data science, and production-support teams. \n

• Familiarity with MLflow, model lifecycle management, model serving, or actuarial-model integration. \n

• Experience implementing data governance, metadata management, lineage, role-based access, audit controls, and regulatory controls. \n

• Familiarity with Azure DevOps, Terraform, Power BI, Microsoft Purview, Azure Monitor, or comparable tools. \n

• Databricks and Microsoft Azure certifications are preferred. \n

• Experience supporting business-critical production platforms with defined availability, recovery, security, and performance requirements. \n

• Good understanding of data-engineering practices, including: \n

• Batch and streaming data ingestion \n

• ETL/ELT pipeline development \n

• Medallion architecture \n

• Data transformations and enrichment \n

• Data quality, validation, and reconciliation \n

• Schema management and data lineage \n

• Spark workload and query optimization \n

• Experience leading globally distributed platform, data-engineering, application, QA, DevOps, and production-support teams. \n

• Ability to participate in architecture reviews, solution-design discussions, code reviews, security assessments, and production-readiness evaluations. \n

• Strong understanding of Agile delivery, release management, risk management, dependency management, and stakeholder governance. \n

\n, * Understanding of reinsurance pricing, underwriting, treaty, and facultative business processes. \n

• Experience integrating Databricks with: \n

• Pricing engines \n

• SQL Server applications \n

• Policy, claims, and accounting systems \n

• Data warehouses and reporting platforms \n

• Internal and third-party data sources \n

• Experience establishing reusable Databricks platform standards, engineering frameworks, and reference architectures. \n

• Experience implementing a hybrid solution of Databricks and SQL

We are seeking an experienced Databricks Platform Engineering Led to support an enterprise-wide pricing application within a global reinsurance organization. The application leverages Databricks as its data and analytics platform and integrates with a SQL Server-based pricing application.