Databricks Integration Lead

This role involves managing integrations across 5 Veeva CRM organizations and 46 integration points, optimizing existing integrations, and ensuring a seamless transition to Vault CRM.

The ideal candidate has deep, hands-on Databricks expertise, strong integration architecture experience, and a background in pharmaceutical or life sciences environments. This is a technical leadership role requiring active involvement in architecture, pipeline development, and engineering team oversight., * Lead end-to-end integration architecture across Databricks, Veeva CRM, Vault CRM, and connected systems.

• Analyze and optimize 46 integration points, identifying opportunities for consolidation and standardization.
• Design scalable data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake.
• Establish standards for medallion architecture, Unity Catalog, workflow orchestration, and CI/CD.
• Develop data mapping and integration strategies to support the Veeva CRM to Vault CRM migration.
• Provide hands-on technical guidance, review pipelines, and troubleshoot complex integration challenges.
• Lead engineering teams, including offshore resources.
• Ensure data validation, governance, regulatory compliance, and successful migration cutover.

• 10+ years of experience in data integration, engineering, or architecture.
• 5+ years of deep, hands-on Databricks experience in enterprise environments.
• Expertise in PySpark, Spark SQL, Delta Lake, Delta Live Tables, Auto Loader, Workflows/Jobs, and Unity Catalog.
• Hands-on experience with Veeva CRM integrations and data models.
• Experience in pharmaceutical or life sciences commercial environments, including HCP/HCO data, call activity, samples, and consent.
• Experience delivering complex, multi-system integration projects.
• Strong API integration experience (REST/SOAP), data pipelines, and performance optimization.
• Proven technical leadership experience, including managing distributed or offshore engineering teams.
• Familiarity with GxP validation and regulated environments.

Preferred Qualifications

• Experience with Vault CRM or Veeva CRM to Vault CRM migrations.
• Databricks certifications.
• Experience with Veeva Network, OpenData, or multi-org integrations.
• Experience with Terraform, Azure DevOps, GitHub Actions, or cloud platforms (AWS, Azure, GCP).