Principal Software Engineer - IT
Dell Technologies Inc.
United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.jobmonkeyjobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Internship / Graduate position
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours
Job source
Tech stack
LangGraph Framework
Artificial Intelligence
Continuous Integration
Python (Programming Language)
Machine Learning
Open Web Application Security
Software Engineering
Large Language Models
Multi-Agent Systems
Langfuse β LLM Observability and Analytics Platform
Job description
- You will own the release gate for AI-native security capabilities, deciding what evidence is required before a workflow can go live.
- You will build evaluation harnesses that measure model and agent behaviour objectively, not by sampling outputs and forming an impression.
- You will turn evaluation into a repeatable, measurable analytics practice so that every workflow after the first costs less to evaluate than the one before it.
You will:
- Define and record acceptance criteria for each capability with the functional owner before evaluation begins, and specify the evidence, ground truth and coverage required to support the claim.
- Build and maintain evaluation harnesses in Python covering dataset construction, test orchestration, scoring and reporting, and integrate them as automated quality gates in CI/CD.
- Design measurable analytics for models and agentic tools - quantifiable metrics for accuracy, consistency across repeated runs, confidence-versus-accuracy calibration, latency and cost - using tooling such as LangFuse for trace-level evaluation and embedding-based similarity scoring for output comparison.
- Own the release gate: apply the criteria to the evidence, record the result, and block releases that do not clear the threshold, with thresholds set proportionate to the consequence of an incorrect agent action.
- Run ongoing regression evaluation after release to detect behavioural drift from model, prompt, data or upstream changes, and report results to engineering and functional stakeholders in decision-ready terms.
Requirements
- 8+ years in software engineering, quality engineering or machine learning evaluation, with systems shipped to production.
- Hands-on construction of evaluation harnesses in Python: dataset design, test orchestration, scoring implementation and result reporting for non-deterministic systems.
- Ability to build measurable analytics of models and agentic tools, using LangFuse (or equivalent LLM observability tooling) and embedding-based comparison techniques.
- Statistical literacy: sample size, variance across runs, and the difference between a real change in behaviour and noise, applied with Python data tooling at volume.
- Experience owning a quality gate or release decision, including declining a release and holding that position under schedule pressure.
Desirable Requirements
- Experience evaluating AI systems within a security domain, and familiarity with agent orchestration frameworks such as LangGraph to understand what is being evaluated.
- Familiarity with the OWASP Top 10 for LLM Applications and the NIST AI Risk Management Framework, ideallyin an audited or regulated environment.
About the company
The Security Resiliency Organization builds the AI-native capabilities that are replacing manual security processes across Dell, and every one of those capabilities has to earn its release through evidence, not demonstration. This team owns the quality bar for that shift: defining what βgood enoughβ means for each workflow and building the harnesses that prove it, before and after release.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.jobmonkeyjobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦