Senior Data Scientist (Model Diagnostics)

STEALTH TALENT SOLUTIONS, INC.
New York, NY, United States
26 days ago
Apply on www.nettemps.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours

Tech stack

Python (Programming Language) Machine Learning Standard Sql Information Technology

Job description

You’ll own the investigative science behind the core prediction models at a mobile advertising technology company: finding where models underperform, figuring out why, and turning that into the roadmap the ML engineering team builds against. This isn’t a model-building or infrastructure role. You’ll diagnose, experiment, and explain, and your work will steer a multi-billion-dollar optimization problem., * Dig into where install-prediction models lose accuracy, slicing by segment, data slice, and campaign type to isolate where and why performance drops

  • Run calibration checks, feature-importance analysis, and slice-level error analysis to turn a vague performance gap into a precise, testable explanation
  • Form hypotheses about model behavior and test them empirically (for example, rerunning a pipeline on a reduced sample to test a data-volume theory)
  • Translate findings into prioritized recommendations for ML engineering, defining the experiment and the expected mechanism, not just the metric
  • Own production investigations end to end: when a campaign’s performance drops, find the root cause (data issues, model shift, prediction drift, privacy-driven signal loss) and report back
  • Surface longer-term opportunities across the model portfolio, quantifying the potential lift and trade-offs
  • Keep technical leadership updated with status and new hypotheses without being asked

Requirements

  • 5-7+ years in data science or applied machine learning, with hands-on model diagnostics work
  • Strong applied statistics and probability, including experiment design
  • Model calibration and error analysis on production models
  • Python and SQL
  • Master’s in data science, statistics, applied math, computer science, or a related quantitative field
  • Comfort explaining why a change moved a metric in terms of weights, loss, and predictions
  • Self-directed: you take a loosely defined problem and drive it to an answer

Nice to have

  • PhD
  • Ad tech or large-scale Real Time prediction systems
  • Class imbalance and feature importance analysis

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.nettemps.com
Prepare application

Good distractions

Loading talks and stories from around this role…