Machine Learning Engineer

Kavaliro
Jacksonville, FL, United States
2 days ago
Apply on jobs.kavaliro.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$145,600 - $156,000
Working hours
Regular working hours

Tech stack

A/B Testing Big Data Cloud Engineering Distributed Data Store Python (Programming Language) Machine Learning Tensorflow Software Deployment Normalized Discounted Cumulative Gain Deep Learning Multi-objective Optimization Data Analytics
+1 more
Data Pipelines

Job description

We are seeking an experienced Machine Learning Engineer with deep expertise in learning-to-rank systems to support a large-scale digital marketplace. This platform connects consumers with service providers and relies heavily on search and recommendation algorithms to drive successful matches.

The organization recently transitioned from a rules-based ranking approach to a neural network powered ranking model. While initial offline evaluation showed promising improvements, production performance revealed more nuanced outcomes across user journeys, conversion stages, and audience segments.

This role focuses on understanding those discrepancies and helping design, build, and launch the next generation ranking system. The work is centered on ranking algorithms, experimentation, evaluation methodologies, and machine learning modeling rather than infrastructure or systems architecture.

Key Challenges

The platform’s search experience involves filtering a large candidate pool and ranking the most relevant results for users. Improving ranking quality requires addressing several complex challenges:

  • Offline-to-online alignment: Traditional offline ranking metrics do not consistently predict real-world business outcomes.
  • Position and selection bias: Training data is generated from previously displayed results, creating biased feedback loops.
  • Multi-objective optimization: Multiple user and business objectives are currently combined into a single ranking target, masking important tradeoffs.
  • Session-aware behavior: Users frequently perform multiple searches before making a decision.
  • Delayed outcomes: Many conversions occur well after the initial search event, resulting in incomplete training labels.
  • Feature utilization: Contextual signals such as device type, channel, and behavioral patterns are not fully leveraged.

What You’ll Do

  • Investigate discrepancies between offline model performance and production results.
  • Improve evaluation frameworks so offline metrics better predict real-world impact.
  • Design stronger experimentation methodologies, including A/B testing strategy and statistical analysis.
  • Apply counterfactual and unbiased learning-to-rank techniques to address position and selection bias.
  • Develop multi-objective ranking approaches that explicitly balance competing success metrics.
  • Incorporate delayed feedback and incomplete conversion signals into model training.
  • Model session-level user behavior and introduce richer contextual features.
  • Train, optimize, and evaluate ranking models on large-scale datasets.
  • Define rollout and validation strategies for production deployment.
  • Collaborate closely with technical stakeholders and mentor internal engineering team members.

Technical Environment

  • Large-scale training datasets containing tens of millions of examples
  • Cloud-native machine learning infrastructure
  • Python-based machine learning stack
  • Deep learning and gradient-boosted ranking frameworks
  • Distributed data storage and model training environments

Requirements

  • Authorization to work in the United States.
  • Hands-on experience building and deploying production learning-to-rank systems for search, recommendations, advertising, or marketplace applications.
  • Strong understanding of ranking methodologies, including pointwise, pairwise, and listwise approaches.
  • Expertise with ranking metrics such as NDCG, MRR, MAP, and related evaluation techniques.
  • Experience addressing position bias, selection bias, and counterfactual learning problems.
  • Strong background in online experimentation, A/B testing, statistical analysis, and performance measurement.
  • Experience with multi-objective optimization and funnel-based ranking systems.
  • Advanced proficiency in Python and modern machine learning frameworks.
  • Ability to communicate technical findings and influence business decisions through data-driven recommendations.

Preferred * Experience working with large-scale marketplace, e-commerce, services, travel, healthcare, or recommendation platforms.

  • Experience modeling delayed feedback, censored labels, or time-dependent conversion outcomes.
  • Experience with session-aware or context-aware ranking systems.
  • Hands-on experience with cloud-based machine learning platforms and large-scale data pipelines.
  • Experience in highly regulated industries.
  • Prior consulting or client-facing experience, including mentoring and knowledge transfer responsibilities.

About the company

By using best practices and optimal employee recruiting strategies, Kavaliro provides employers with employment solutions by providing the most qualified and professional employees, who can staff both project and permanent positions in order to ensure the ongoing success of all types of businesses. We use a streamlined-yet-thorough approach to staffing that saves our clients administrative time, resources and money.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on jobs.kavaliro.com
Prepare application

Good distractions

Loading talks and stories from around this role…