Principal Engineer - Agentic AI for IT Operations
Role details
Job location
Tech stack
Job description
We are seeking a Principal Engineer to lead the design, implementation, and adoption of agentic AI solutions that transform how our Corporate Technology organization manages incidents, changes, and problems across complex, production environments including SaaS, COTS and homegrown applications which are on-prem and the cloud. This role is a key technical leader driving operational reliability, automation, and risk-aligned engineering practices across the enterprise. You will work closely with Security, Enterprise Operations, Risk and Compliance, and application teams to architect solutions, ensure compliance with mandates, and accelerate delivery. You'll bring deep technical expertise, a collaborative mindset, and a passion for modernizing IT operations with AI-driven automation. At this time, Ally will not sponsor a new applicant for employment authorization for this position.erations.
The Work Itself
Agentic AI Engineering & Architecture
- Architect, develop, and evolve agentic AI solutions that automate incident management, change management, and problem management workflows.
- Build intelligent agents for detection, triage, diagnostics, communication, and remediation across production applications for SaaS, COTS and homegrown applicationsDesign and implement scalable integration patterns with monitoring, logging, ticketing, automation, and observability platforms.
Operational Excellence & Production Support
- Partner with production support, SRE, and application teams to improve reliability, reduce MTTR, and automate operational runbooks.
- Establish engineering standards for safe deployment, testing, and monitoring of AI-enabled automation.Provide technical leadership during major incidents, ensuring agent-driven tooling and guidance supports consistent recovery.
Governance, Risk, and Compliance Leadership
- Drive completion of corporate tech risk and compliance mandates across application teams.
- Define technical controls, ensure alignment with regulatory requirements, and guide teams through implementation.Track program execution, identify gaps, and escalate risks with clarity and data.
Cross-Functional Partnership
- Serve as a key technical liaison across Security, Enterprise Operations, Infrastructure, Risk, and Compliance.
- Lead technical discussions, influence architectural decisions, and gain alignment across multiple stakeholder groups.Communicate complex concepts in a clear, approachable, and outcomes-focused way.
Leadership & Team Culture
- Mentor engineers, champion best practices, and set a high bar for engineering quality and accountability.
- Foster a collaborative, approachable, team-first culture-modeling communication, ownership, and transparency.
- Help shape long-term strategies for AI-driven IT operations across the enterprise.
Requirements
Do you have experience in Stakeholder management?, Do you have a High school diploma or GED?, * 7+ years of relevant experience or equivalent combination of education and experience
- High School Diploma or GED equivalent, * 5+ years software engineering experience with strong depth in distributed systems, enterprise operations, or SRE.
- Bachelors Degree in Computer Science or similar field or equivalent experience.
- Hands-on experience building automation or AI-driven operational tooling (e.g., agent-based systems, LLM apps, observability-driven automation, self-healing systems).
- Expertise with incident, change, and problem management processes in large enterprise environments.
- Strong understanding of governance, risk, and compliance frameworks (technology control standards, audit readiness, policy implementation).
- Proven ability to lead cross-functional technical initiatives with diverse partners.
- Excellent communication skills; approachable, practical, and team-oriented working style.
- Ability to influence without authority and drive execution at scale.
Highly Preferred Skills:
- Awareness the latest on AI technology
- Experience with LLM frameworks, multi-agent orchestration, or retrieval-augmented automation.
- Experience with Claude Code, IDEs with AI agents like Cursor, VSCode with Agents etc..
- Familiarity with ServiceNow, APM/log platforms (Dynatrace, Splunk) CI/CD, and cloud platforms.
- Background in FinTech or regulated industries.
Benefits & conditions
Pulled from the full job description
- Childcare
- Tuition reimbursement
- Paid parental leave
- Employee stock purchase plan
- Parental leave
- Health insurance
- 401(k) matching, Ally's compensation program offers market-competitive base pay and pay-for-performance incentives (bonuses) based on achieving personal and company goals. Our Total Rewards program includes industry-leading compensation and benefits plus additional incentives that are designed to meet your needs and those of your family so you can get the most out of your career and your life, including:
- Time Away: Program starts at 20 paid time off days in addition to 11 paid holidays and 8 hours of volunteer time off yearly (time off days are prorated based on start date and program varies based on full or part-time status and management level).
- Planning for the Future: plan for the near and long term with an industry-leading 401K retirement savings plan with matching and company contributions, student loan pay downs and 529 educational save up assistance programs, tuition reimbursement, employee stock purchase plan, and financial learning center and financial coach access.
- Supporting your Health & Well-being: flexible health and insurance options including medical, dental and vision, employee, spouse and child life insurance, short- and long-term disability, pre-tax Health Savings Account with employer contributions, Healthcare FSA, critical illness, accident & hospital indemnity insurance, and a total well-being program that helps you and your family stay on track physically, socially, emotionally, and financially.
- Building a Family: adoption, surrogacy and fertility assistance as well as paid parental and caregiver leave, Dependent Day Care FSA back-up child and adult/elder care days and childcare discounts.
- Work-Life Integration: other benefits including Mentally Fit Employee Assistance Program, subsidized and discounted Weight Watchers® program and other employee discount programs.