> Markdown version of [/jobs/ext/4354478-senior-data-scientist-model-diagnostics](https://www.wearedevelopers.com/jobs/ext/4354478-senior-data-scientist-model-diagnostics). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Scientist (Model Diagnostics) - **Company:** STEALTH TALENT SOLUTIONS, INC. - **Location:** New York, NY, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Python (Programming Language), Machine Learning, Standard Sql, Information Technology - **Published:** October 22, 2026 - **Apply:** https://www.nettemps.com/us/en/W513B6E5C1DF2C84C6B.ntap ## About the Role * 5-7+ years in data science or applied machine learning, with hands-on model diagnostics work * Strong applied statistics and probability, including experiment design * Model calibration and error analysis on production models * Python and SQL * Master's in data science, statistics, applied math, computer science, or a related quantitative field * Comfort explaining why a change moved a metric in terms of weights, loss, and predictions * Self-directed: you take a loosely defined problem and drive it to an answer Nice to have * PhD * Ad tech or large-scale Real Time prediction systems * Class imbalance and feature importance analysis ## Description You'll own the investigative science behind the core prediction models at a mobile advertising technology company: finding where models underperform, figuring out why, and turning that into the roadmap the ML engineering team builds against. This isn't a model-building or infrastructure role. You'll diagnose, experiment, and explain, and your work will steer a multi-billion-dollar optimization problem., * Dig into where install-prediction models lose accuracy, slicing by segment, data slice, and campaign type to isolate where and why performance drops * Run calibration checks, feature-importance analysis, and slice-level error analysis to turn a vague performance gap into a precise, testable explanation * Form hypotheses about model behavior and test them empirically (for example, rerunning a pipeline on a reduced sample to test a data-volume theory) * Translate findings into prioritized recommendations for ML engineering, defining the experiment and the expected mechanism, not just the metric * Own production investigations end to end: when a campaign's performance drops, find the root cause (data issues, model shift, prediction drift, privacy-driven signal loss) and report back * Surface longer-term opportunities across the model portfolio, quantifying the potential lift and trade-offs * Keep technical leadership updated with status and new hypotheses without being asked