Data Platform Lead (Hybrid)

Platform Strategy & Roadmap Execution

• Serve as the platform owner for AMA's enterprise Databricks lakehouse, accountable for its technical direction, delivery, and operational health.
• Translate the enterprise data platform strategy and multi-year roadmap into an actionable technical delivery plan and engineering backlog.
• Define and maintain the platform's technical architecture, including workspace topology (development, staging, production), Delta Lake design, and Unity Catalog structure.
• Maintain a prioritized platform engineering backlog using agile delivery methods, balancing new capability development, technical debt remediation, and operational work.
• Evaluate emerging Databricks and data ecosystem capabilities and recommend adoption where they advance reliability, cost efficiency, or business value.
• Contribute to platform budget planning and forecasting, and provide technical input into platform-related procurement and vendor management.

Platform Architecture & Engineering Delivery

• Owns platform prioritization and backlog, including intake and sequencing of requests from business units and engineering teams. Own the technical onboarding of new data sources and business-unit workloads onto the platform.
• Collaborate with Sr Data Engineer to lead the design, build, and operation of data ingestion, transformation, and delivery pipelines across batch and streaming workloads.
• Ensure adherence engineering standards for data modeling, code quality, version control, testing, and CI/CD across the platform.
• Design and maintain self-service analytics capabilities and reusable data products that reduce duplicated effort across business units.
• Provide and operate shared MLOps platform capabilities (e.g., model registry, deployment frameworks), enabling data science and engineering teams to operationalize models.
• Develop and maintain platform documentation, runbooks, and standard operating procedures.
• Serve as the final decision authority on platform architecture and engineering standards; provide design approvals and guardrails for data engineering teams.

Platform Reliability & Governance Implementation

• Contribute to the reliability, performance, scalability, and security of the enterprise data platform, in partnership with the Director II and both IT and federated business-unit engineering teams.
• Define and operate observability tooling and monitoring practices for platform health, cost optimization, model performance, data quality, and governance compliance; drive continuous improvement initiatives.
• Drive cloud cost optimization through cluster configuration, capacity planning, storage management, and cost allocation across business units.
• Implement platform security controls, including encryption, access management, and audit logging, in alignment with IT Security policy and HIPAA and other regulatory requirements
• Define, configure, and enforce platform-level governance controls (e.g., Unity Catalog, access policies, lineage tooling, data quality monitoring, validation, and issue-escalation processes).
• Partner with the Data Governance Lead to translate Enterprise Data Office policies into platform configuration and automated enforcement.
• Implement technical controls supporting AI governance, including model lifecycle management, monitoring, and drift detection.

May include other responsibilities as assigned

1. Bachelor's degree in computer science, engineering, information systems, or related field preferred or equivalent work experience and HS diploma/equivalent education required
2. 7+ years of experience in data engineering, data platform engineering, or data architecture required.
3. Proven experience designing, building, and operating a large-scale data platform, preferably built on modern lakehouse architectures (Databricks preferred)
4. Proven expertise with Delta Lake, Unity Catalog, Apache Spark, and SQL-based transformation frameworks (dbt or equivalent)
5. Experience owning platform reliability, performance tuning, observability, and cloud cost optimization
6. Proven experience implementing data governance controls, including access, data quality, metadata, and lineage, within a data platform
7. Working knowledge of AI/ML workflows, MLOps practices, and model lifecycle management
8. Experience operating within a regulated data environment (healthcare / HIPAA, financial services, or similar) preferred
9. Demonstrated success leading technical teams and collaborating across engineering, security, and business disciplines
10. Strong ability to influence, negotiate, and drive alignment across stakeholders without direct authority
11. Ability to balance innovation and delivery speed with operational reliability, cost efficiency, and regulatory requirements

The American Medical Association (AMA) is the nation's largest professional Association of physicians and a non-profit organization. We are a unifying voice and powerful ally for America's physicians, the patients they care for, and the promise of a healthier nation. To be part of the AMA is to be part of our Mission to promote the art and science of medicine and the betterment of public health. At AMA, our mission to improve the health of the nation starts with our people. We foster an inclusive, people-first culture where every employee is empowered to perform at their best. Together, we advance meaningful change in health care and the communities we serve. We encourage and support professional development for our employees, and we are dedicated to social responsibility. We invite you to learn more about us and we look forward to getting to know you. We have an opportunity at our corporate offices in Chicago for a Data Platform Lead (Hybrid) on our Information Technology team. This is a hybrid position reporting into our Chicago, IL office, requiring 3 days a week in the office. This role will serve as the platform owner for AMA's new enterprise Databricks data platform, leading its architecture, engineering, and day-to-day operation in support of AMA's mission to promote the art and science of medicine and the betterment of public health. This role is responsible for the secure, reliable, performant, and cost-effective operation of the platform. This role translates enterprise data platform strategy and governance frameworks into technical implementation, and partners across IT, security, and federated business-unit engineering teams to enable responsible data and AI usage. This role serves as the primary technical authority for AMA's lakehouse environment. This role is accountable for platform capabilities, standards, and operations