Senior Data Scientist + Gen AI
AIT Global, Inc.
Mountain View, CA, United States
3 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Airflow
Amazon Web Services
Confluence
JIRA
Microsoft Azure
Cloud Computing
Python (Programming Language)
Machine Learning
Role-Based Access Control
OpenAI
Management of Software Versions
+10 more
Workflow Management Systems
Retrieval-Augmented Generation
Large Language Models
Prompt Engineering
Hallucination Detection
Llamaindex
Generative AI
Agentic-AI
Low Latency
Drift Detection
Job description
Prompt engineering (system prompts, tool prompts, function calling, versioning with evaluations) Evaluation & observability, including:
- Ground truth setup
- Confusion metrics
- Hallucination rate measurement
- LLM-as-judge with human review
- Cost & latency monitoring
- Retrieval strategies & prompt patterns
- Context management
- Hallucination mitigation
- Faithfulness & grounding validation
- Agentic workflows
- Multi-step reasoning
- Tool orchestration
- Agent success rate measurement
- LangChain / LlamaIndex (or equivalent) proficiency
- Cloud LLM providers
- Azure OpenAI
- AWS Bedrock
- Vertex AI
- Workflow orchestration
- Airflow
- Dagster
- Security & privacy
- PII handling
- RBAC
- Strong Python expertise with real production experience
- Success Criteria
- Production-ready prompt + RAG system
- Measurable quality lift (accuracy, faithfulness, confusion metrics)
- Reduced hallucinations
- Repeatable evaluation harness
- Observable system (quality, latency, cost, drift monitoring)
Requirements
- Large Language Models, AWS, Prompt Engineering, Python, Machine Learning, Gen AI, Confluence, Jira
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦