> Markdown version of [/jobs/ext/2722675-data-scientist](https://www.wearedevelopers.com/jobs/ext/2722675-data-scientist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # data scientist - **Company:** Paradigm - **Location:** United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Clinical Data Repository, Code Review, Data Infrastructure, Statistical Hypothesis Testing, Python (Programming Language), Operational Databases, SQL Databases, Management of Software Versions, Large Language Models, Pyspark, Information Technology, Tools for Reporting, Software Version Control, Databricks - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/senior-data-scientist-llm-evaluation-business-intelligence-paradigm-health-8793444 ## About the Role * Bachelor's degree or higher in data science, statistics, biostatistics, computer science, mathematics, epidemiology, or a related field. * 4+ years of experience as a data scientist, analytics engineer, ML/data professional, or other highly analytical role in life sciences, biotech, healthcare, or a related industry. * Solid understanding of how production LLM pipelines work, and hands-on experience evaluating LLM or NLP systems - accuracy metrics, error analysis, RAG/retrieval quality, prompt evaluation, or similar. * Strong proficiency in SQL, with experience building production data models in dbt (or a comparable transformation framework). * Proficiency in at least one programming language (Python preferred; PySpark a plus). * Experience with modern data platforms such as Databricks, and BI/analytics tools such as Hex, or similar. * Fluency with data science best practices: reusability, version control, documentation, testing, code review, and reproducibility. * Excellent written and verbal communication, with the ability to translate technical concepts into actionable insights for non-technical stakeholders and to develop reporting logic collaboratively with them. * Comfort with AI tooling and agentic workflows in day-to-day work. * Comfort with ambiguity and adaptability to a mission-driven, fast-paced startup environment. Bonus * Master's degree or higher in a quantitative field. * Experience establishing or working within a semantic layer for organization-wide reporting. * Experience integrating LLM-based workflows into production systems, and working closely with ML/AI engineering teams. * Familiarity with clinical data elements (oncology a plus) and the clinical trial industry or research operations. * Strong statistical knowledge, including regression, classification, hypothesis testing, causal inference, or Bayesian methods (e.g., continuous toxicity monitoring for trial protocols). * Prior experience mentoring other data scientists or leading cross-functional projects (for senior candidates). ## Description As a Data Scientist on our team, you will help define how we measure and trust the LLM systems that power Paradigm's clinical data products and build the business intelligence layer that turns that data into reliable reporting across the org. You'll work at the intersection of applied AI evaluation, analytics engineering, and clinical research, partnering with AI engineers, data engineers, clinicians, and non-technical stakeholders. In this role you'll be responsible for delivering reliable data that helps Paradigm measure its business and drive outcomes. You'll partner with teams across the organization to define requirements, then research and build the models and reporting that meet them. This is a deeply collaborative, internally-facing role: your partners range from highly technical teammates to non-technical business, clinical, and product stakeholders, and much of your impact comes from translating between them - turning ambiguous questions into well-defined data products and communicating results, and their caveats, clearly to every audience. Your primary focus will be evaluating production LLM pipelines: designing accuracy metrics, measuring retrieval effectiveness, and making principled cost/quality tradeoffs for the prompts and models that extract clinical meaning from patient data. Alongside this, you'll help establish a trustworthy business intelligence foundation - production data models, a shared semantic layer, and governed pipelines that give the whole organization consistent, defensible numbers. It's a role for someone who wants to bring scientific rigor to a fast-moving AI system and see their work drive measurable impact for patients and providers. What you'll do: * Design and run evaluations of production LLM pipelines, developing accuracy and quality metrics that tell us how well our systems perform on real clinical tasks. * Support evaluation of our Patient Trial Evaluation (PTE) pipeline - assessing prompts and trial-matching logic for accuracy in reporting, and surfacing where and why they fail. * Measure and improve the effectiveness of our retrieval-augmented generation (RAG) systems, from retrieval quality through final output. * Use accuracy metrics to inform cost/quality tradeoffs across trials, prompts, and models, giving the team a clear basis for which approaches to ship. * Establish and maintain a library of "best prompts" for recurring clinical concepts, backed by evidence rather than intuition. * Partner with AI Engineering to close the loop between evaluation findings and production improvements. * Build and maintain production-grade data models using SQL and dbt on Databricks, ensuring analytics logic is reliable, maintainable, and production-ready. * Help establish and grow a semantic layer that enables consistent, trusted reporting across the organization. * Advance data governance practices across dbt, Databricks, and Hex - documentation, testing, versioning, and clear ownership of metrics. * Collaborate with non-technical internal stakeholders to develop reporting logic, answer data questions, and translate business needs into well-defined data products. * Support both internal and external reporting needs with accurate, well-tested datasets. * For senior applicants: mentor and support junior data scientists - providing guidance on evaluation and analytics approaches, technical best practices, and professional growth. Who you are You are a rigorous, versatile data scientist who is energized by making AI systems measurable and trustworthy. You care about getting the number right and being able to defend it. You're equally comfortable designing an evaluation framework for an LLM pipeline and writing the SQL and dbt models that power reliable reporting. You bring a strong interest in clinical research and a passion for advancing healthcare through data innovation. You're autonomous and driven - you can take an ambiguous problem and run with it, structuring the approach yourself rather than waiting for a fully specified spec. You're curious, collaborative, and hold yourself to data science best practices as a default, not an afterthought. ## Related Videos - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source) - [Are Code Reviews Worth It? Insights from 16 Years of Review Data](https://www.wearedevelopers.com/videos/1135-are-code-reviews-worth-it-insights-from-16-years-of-review-data) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)