Principal Site Reliability Engineer

Arcadia Corporation
United States
3 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$180,000.0 - $230,000.0
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Systems Engineering Audit Trail Code Review Software Debugging Distributed Data Store Identity and Access Management Python (Programming Language) Key Management Load Testing Octopus Deploy Role-Based Access Control
+9 more
Reliability Engineering Data Processing Cloud Platform System Apache Spark Mttr Amazon Virtual Private Cloud (VPC) Kubernetes Cassandra Terraform

Job description

Arcadia’s customers rely on us to securely process and deliver high-value healthcare insights. Reliability, availability, performance, and security are foundational to trust-especially when systems support critical workflows and handle PHI. As a Principal Site Reliability Engineer, you’ll set reliability strategy across teams, drive cross-cutting platform improvements, and ensure we can scale delivery without scaling operational burden. What Success Looks Like In 3 months

Build deep context on Arcadia’s platform, production risks, and operational practices. Participate in on-call/incident response and quickly improve signal quality for at least one critical domain (dashboards, alerts, traces, runbooks). Identify a high-leverage reliability initiative and align stakeholders on scope, success metrics, and milestones.

In 6 months

Establish SLOs/error budgets for key customer journeys, drive operational readiness standards for launches, and lead remediation for recurring incidents with measurable reductions in customer impact and MTTR. Deliver major toil-reduction improvements via automation and self-service workflows.

In 12 months

Own and execute a reliability program with cross-org impact (e.g., GitOps delivery guardrails, observability platform evolution, resilience/DR improvements, or secure infrastructure controls). Influence architecture decisions, establish org-wide operational standards, and mentor Staff engineers-raising the reliability and security bar across Arcadia., * Act as the technical leader for reliability for one or more domains; set direction and standards while remaining hands-on where it matters most

  • Drive reliability strategy across critical services: define SLOs/SLIs, error budgets, and reliability KPIs aligned to customer journeys and outcomes
  • Own incident response maturity: lead complex incidents, improve incident command practices, and ensure high-quality RCAs with prioritized, tracked remediation
  • Architect and implement automation to reduce toil and risk: runbook automation, self-service tools, and safe operational workflows (Python + Argo Workflows)
  • Advance GitOps delivery practices using Argo CD: promotion strategies, progressive delivery/canaries, and guardrails that reduce deploy risk
  • Scale infrastructure management with Crossplane and Terraform: reusable patterns, policy controls, and paved roads for teams
  • Lead operational readiness and reliability reviews for new features/architectural changes; reinforce non-functional requirements (availability, latency, security, cost)
  • Improve performance and cost efficiency through capacity planning, load testing, right-sizing, and architecture recommendations across AWS services
  • Champion infrastructure security best practices for environments that handle PHI (least privilege, secrets management, auditability, and defense-in-depth)
  • Mentor Staff and Senior engineers through design reviews, code reviews, pairing, and documentation; raise reliability standards across teams, * Customer Focus: champions customer impact; drives SLO definition with product partners; participates in incidents to limit customer impact; may engage customers to understand problems
  • Technical Leadership: leading cross-team technical representative; negotiates interfaces; anticipates edge cases; designs telemetry for availability and reliability
  • Total Ownership: owns outcomes from requirements and design through production support; transitions complex changes with multi-phase rollouts and long-term ownership
  • Effective Communication: communicates to diverse audiences; finalizes key documentation (runbooks, guides, FAQs); synthesizes standards and best practices
  • Proactive Leadership: coaches senior/peer teams primarily through review; delegates appropriately; sets clear expectations (Definition of Done) and improves service processes/rotations, This position is responsible for following all Security policies and procedures in order to protect all PHI under Arcadia’s custodianship as well as Arcadia Intellectual Properties. For any security-specific roles, the responsibilities would be further defined by the hiring manager. About Arcadia Arcadia.io helps innovative providers and payers across the country transform healthcare to reduce cost while improving patient health. We do this by aggregating large amounts of disparate data, applying algorithms to identify opportunities to provide better patient care, and making those opportunities actionable by physicians at the point of care in near-real time. We are passionate about helping our customers drive meaningful outcomes. We are growing fast and have emerged as a market leader in the highly competitive population health management software market and have been recognized by industry analysts KLAS, IDC, Forrester, and Chilmark for our leadership. For a better sense of our brand and products, please explore our website. Protect Yourself If you have concerns about the authenticity of a job offer or recruitment-related communication claiming to be from Arcadia, we encourage you to verify by contacting us directly at (781) 202-3600 and select option 3. For more information, visit our website. This position is responsible for following all Security policies and procedures in order to protect all PHI under Arcadia’s custodianship as well as Arcadia Intellectual Properties. For any security-specific roles, the responsibilities would be further defined by the hiring manager. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us. You must create an Indeed account before continuing to the company website to apply

Requirements

  • 8+ years of experience in SRE, platform engineering, systems engineering, or related roles operating production services at scale
  • Demonstrated principal-level impact: leading cross-team initiatives, influencing architecture decisions, and driving sustained improvements in reliability and operations
  • Expertise in Kubernetes operations and troubleshooting, including safe rollout/rollback patterns, workload debugging, and operational guardrails
  • Strong GitOps experience with Argo CD; experience building delivery workflows and automation using Argo Workflows
  • Strong infrastructure orchestration and provisioning experience with Crossplane and Terraform; ability to define reusable platform patterns and controls
  • Deep AWS experience (IAM, networking/VPC, compute, storage, managed services, observability) and strong understanding of reliability and failure modes in cloud systems
  • Proficiency in Python for building automation, tooling, and reliability improvements
  • Strong incident management and on-call leadership experience, including measurable improvements (availability, MTTR, alert quality, cost, or operational maturity)
  • Excellent communication skills: can translate technical risk and reliability tradeoffs to engineering leadership, product, and stakeholders; produces high-quality docs/runbooks, * Experience with ScyllaDB or similar distributed databases (e.g., Cassandra) and their reliability/performance characteristics
  • Experience with Spark or data processing platforms, including reliability and cost considerations for large-scale workloads
  • Familiarity with agentic coding practices and principles (safe automation, reviewable changes, guardrail-first workflows)
  • Strong infrastructure security knowledge: threat modeling for cloud/Kubernetes, RBAC/IAM design, secrets management, supply chain security, and security observability

Benefits & conditions

  • Be a part of a mission driven company that is transforming the healthcare industry by changing the way patients receive care
  • A flexible, remote friendly company with personality and heart
  • Employee driven programs and initiatives for personal and professional development
  • Become a member of the talented, energized, diverse and purpose-driven Arcadian Community

About the company

Arcadia is dedicated to happier, healthier days for all. We believe that there is a better healthcare world - one powered by data. Our platform transforms complex, diverse data into a unified foundation for health, helping organizations deliver better care, boost revenue, and lower costs.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

1:34 min

Pivoting careers into specialized platform engineering roles

Xavier Portilla Edo · LIVE

3:08 min

Scaling semantic search with Astra DB and Apache Cassandra

David Leconte David Leconte +1 · WWC 2024

1:34 min

Essential commands for running and testing Terraform configurations

Hennie Francis · LIVE

3:08 min

Aligning engineering processes with core business impact metrics

Chris Riley · WWC 2021

3:22 min

Introduction to building real-time generative agents

Dieter Flick · WWC 2023

2:32 min

Overview of Terraform and Terraform Cloud features

Devlin Duldulao · LIVE

Videos

See all

Related articles

See all