Staff GenAI Engineer - Application Performance Monitoring (APM)

Datadog
New York, NY, United States
3 months ago
Apply on indeed.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Compensation
$234,000.0
Working hours
Regular working hours
Job source

Tech stack

Adobe InDesign Application Performance Management Profiling Machine Learning Datadog Dynatrace

Job description

We’re looking for a Staff Software Engineer with deep experience in GenAI/ML to join Datadog’s Application Performance Monitoring (APM) team. APM is a product which provides deep visibility into applications, enabling users to identify performance bottlenecks, troubleshoot issues, and optimize services. With distributed tracing, profiling, out-of-the-box dashboards, and seamless correlation with other telemetry data, Datadog APM provides some of the deepest and most structured visibility into the health and performance of applications. This context sets us up for an opportunity to be the world leaders in agentic investigations and incident troubleshooting., You’ll act as a technical leader within the APM group, focused on agentic workflows. You’ll lead efforts to design, train, evaluate, and deploy GenAI/ML models at scale. We’re looking for a product-minded ML engineer with strong technical expertise, excellent communication skills, and a track record of driving impactful initiatives end to end.

At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

What You’ll Do:

  • Act as a technical leader within the APM organization, driving GenAI/machine learning projects from concept to production.
  • Build and benchmark GenAI/ML models using state-of-the-art techniques.
  • Collaborate with cross-functional teams to build automated investigation and triaging tools.
  • Influence product direction by bringing a strong product mindset to your work, always advocating for the end user.
  • Guide teams through ambiguity, scaling challenges, and evolving requirements with clear technical direction.
  • Actively mentor engineers and influence engineering culture through leadership in design reviews, technical talks, and working groups.

Requirements

Do you have experience in Triage?, Do you have a Bachelor’s degree?, * You have a BS/MS/PhD in a scientific field or equivalent experience

  • 10+ years of relevant engineering experience, as well as experience acting as a technical lead
  • Proven track record of leading large-scale GenAI/ML initiatives in a product-driven environment
  • You have significant experience in model deployment, development, training, fine-tuning, or evaluation
  • Ability to drive initiatives across cross-functional teams, and solve ambiguous challenges

Benefits & conditions

  • Get to build tools for software engineers, just like yourself. And use the tools we build to accelerate our development.
  • Have a lot of influence on product direction and impact on the business.
  • Work with skilled, knowledgeable, and kind teammates who are happy to teach and learn
  • Competitive global benefits
  • Continuous professional development

Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.

LI-Hybrid

About the company

Datadog is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infrastructure, data, models, and security into one place, using AI to detect and resolve issues before they impact customers. Trusted globally by Fortune 500 companies and high-growth AI leaders, Datadog enables businesses to move faster with clarity and confidence. Learn more about #DatadogLife on Instagram, LinkedIn, and Datadog Learning Center.

Equal Opportunity at Datadog:

Datadog is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and other characteristics protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. Here are our Candidate Legal Notices for your reference.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:46 min

Integrating AI into web performance engineering workflows

Perf + AI

1:01 min

Connecting frontend application performance to user retention and revenue

Dani Coll Dani Coll · World Congress 2025

47 sec

Profiling native execution calls with async-profiler

Gonzalo Ortiz Jaureguizar Gonzalo Ortiz Jaureguizar · World Congress 2026 Europe

1:36 min

Visualizing memory limits and isolating suspicious endpoints

Dina Matveev Dina Matveev · Europe 2026 Virtual

1:08 min

Analyzing error logs and root causes using artificial intelligence

Nishil Patel Nishil Patel · World Congress 2025

2:08 min

Essential engineering roles in the generative AI space

Mary Grygleski Mary Grygleski · LIVE

Videos

See all

Related articles

See all