> Markdown version of [/jobs/ext/3438224-splunk-observability-engineer](https://www.wearedevelopers.com/jobs/ext/3438224-splunk-observability-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Splunk Observability Engineer

- **Company:** System One
- **Location:** Columbia, SC, United States
- **Salary:** $90,000.0 - $160,000.0
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Amazon Elastic Compute Cloud, Continuous Integration, Github, JSON, Python (Programming Language), Software Troubleshooting, Gitlab, Cloudwatch, Terraform, Splunk
- **Published:** September 10, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=98c2a5d5902a42ba

## About the Role

* Strong Splunk admin experience in large production setups (forwarders, indexers, search heads)
 * Advanced SPL skills
 * Strong AWS experience: EC2, ALB, CloudWatch
 * Hands on OpenTelemetry experience, including Splunk integration
 * Python for automation
 * GitLab, GitHub, Terraform Enterprise, CI/CD, JSON Strong troubleshooting and clear communication skills
*

## Description

Hands on role supporting large Splunk and observability platforms in an AWS production setting.

 * Run, upgrade and support Splunk Enterprise on AWS (EC2)
 * Manage the full data lifecycle: ingestion, indexing, search, dashboards, alerts, retention and recovery
 * Write and tune complex SPL queries
 * Build dashboards for monitoring and incident triage
 * Fix ingestion failures, missing data, volume spikes and outages
 * Restore and replay data after interruptions
 * Reduce noise, storage use and cost
 * Connect Splunk with CloudWatch and OpenTelemetry
 * Build and maintain OpenTelemetry agent and collector packages
 * Automate tasks with Python
 * Handle patching and security fixes
 * Lead troubleshooting calls across teams
 * Take part in on call rotation


## Related Videos

- [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json)
- [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next)
- [GitLab CI pipelines for a whole company](https://www.wearedevelopers.com/videos/143-gitlab-ci-pipelines-for-a-whole-company)
- [Better Together: Leveraging Your Observability Tools as a SIEM](https://www.wearedevelopers.com/videos/2118-better-together-leveraging-your-observability-tools-as-a-siem)
- [Enabling automated 1-click customer deployments with built-in quality and security](https://www.wearedevelopers.com/videos/83-enabling-automated-1-click-customer-deployments-with-built-in-quality-and-security)
- [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure)

## Related Articles

- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Dev Digest 131 - AI'm not sure about OSS](https://www.wearedevelopers.com/magazine/472-dev-digest-131-ai-m-not-sure-about-oss)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)