Lead Product Engineer - Observability (Enterprise App Monitoring) - Remote

Allstate is seeking a forward-thinking Lead Product Engineer to drive innovation in its AI-powered Observability platform. This role sits at the intersection of AI engineering, observability, and software development, with a strong focus on automation, intelligent insights, and self-healing systems across hybrid and multi-cloud environments.

You will lead the design and development of next-generation observability solutions powered by agentic AI, enabling proactive detection, diagnosis, and remediation of issues. This role emphasizes building scalable platforms, intelligent automation, and developer-centric tooling that enhance reliability, performance, and operational efficiency.

As a technical leader, you will shape product strategy, drive engineering excellence, and deliver AI-driven monitoring and automation capabilities that transform how applications and platforms are operated across the enterprise. Key Responsibilities AI & Intelligent Automation

• Design and build agentic AI solutions for observability, including autonomous agents capable of anomaly detection, root cause analysis, and automated remediation.
• Embed AI/ML capabilities into observability platforms to enable predictive insights, anomaly detection, and intelligent alerting.
• Develop AIOps frameworks that reduce noise, automate workflows, and improve incident response times.
• Leverage LLMs and AI orchestration frameworks to create self-service diagnostics and operational assistants.
• Architect and implement end-to-end observability solutions across logs, metrics, traces, and events.
• Enhance observability platforms (Datadog, Dynatrace, New Relic, AppDynamics, OTEL) with custom integrations, automation, and AI enhancements.
• Build real-time insights and health analytics to ensure system reliability, scalability, and performance.
• Lead development of scalable, reusable platform components and APIs for observability and automation.
• Build developer-first tools and frameworks that simplify instrumentation, monitoring, and diagnostics.
• Apply modern software engineering practices including TDD, CI/CD, microservices, and cloud-native design.
• Develop full-stack solutions (backend services, dashboards, automation tooling) using Java/Spring Boot, Python, Node.js, and React.
• Drive automation-first strategies to eliminate manual operations and improve efficiency.
• Build self-healing systems that automatically detect and remediate issues.
• Lead initiatives in infrastructure-as-code, Observability-as-code and automated deployment pipelines.
• Engineer highly resilient systems by focusing on performance, scalability, and fault tolerance.
• Define and evolve enterprise observability architecture, incorporating AI and automation as core principles.
• Collaborate with architecture, platform, and security teams to align solutions with enterprise standards.
• Champion standardization, reusable patterns, and platform scalability across the organization.
• Act as a technical leader and mentor, guiding teams in AI, observability, and engineering best practices.
• Partner with cross-functional teams to deliver integrated, high-impact platform solutions.
• Communicate technical concepts and strategies effectively to both technical and executive audiences.
• Foster a culture of innovation, automation, and continuous improvement., The candidate(s) offered this position will be required to submit to a background investigation.

Joining our team isn't just a job - it's an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. One where you can shape the future of protection while supporting causes that mean the most to you. Joining our team means being part of something bigger - a winning team making a meaningful impact.

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.

• 5+ years of engineering experience building software, platforms, or automation solutions, with a strong emphasis on observability and distributed systems.
• 4+ years of hands-on experience with observability platforms (Datadog, Dynatrace, OTEL).
• 3+ years of software development experience using Java (Spring Boot), Python, Node.js, or React.
• 2+ years of hands-on experience building agentic AI systems (autonomous agents, AI workflows, or AI-native applications).
• Proven ability to design and implement AI-driven automation and AIOps solutions.
• Experience integrating LLMs, orchestration frameworks, or AI pipelines into production systems.
• Strong experience with Kubernetes and cloud-native architectures.
• Hands-on experience in hybrid environments (on-prem + cloud) across Linux and Windows.
• Expertise in API development, event-driven systems, and microservices architecture.
• Familiarity with CI/CD pipelines, infrastructure-as-code, and DevOps practices.
• Strong problem-solving skills with a focus on automation and innovation.
• Excellent communication skills to articulate complex technical solutions.
• Passion for emerging technologies, particularly AI in observability and operations., Agentic AI, Agentic AI, Agile Methodology, AI Agents, AI Ops, Anomaly Detection, API Development, AppDynamics, Application Monitoring, Application Programming Interface (API), Automation, Automation Solutions, Back-End Development, Building Architecture, Business, Business Direction, Business Objectives, Business Operations, Business Processes, Business Software, C (Programming Language), Cloud Infrastructure, Cloud Native, Collaboration, Collaborative Development {+ 65 more}, When working from home, you must have a dedicated, private workspace free from distractions, along with appropriate desk and seating. Reliable internet is required, with minimum speeds of 50 MB download and 5 MB upload.

Compensation offered for this role is 100,000.00 - 170,500.00 annually and is based on experience and qualifications.

At Allstate, great things happen when our people work together to protect families and their belongings from life's uncertainties. And for more than 90 years, our innovative drive has kept us a step ahead of our customers' evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection., Allstate provides a comprehensive technology setup, including a laptop, monitors, headset, keyboard, and mouse. Employees eligible to work from home also receive a monthly connectivity reimbursement to help offset internet costs.