Splunk Monitoring & Observability Engineer

DTEL Engineering & Consultants Inc
Washington, DC, United States
8 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source

Tech stack

Application Programming Interfaces (APIs) Amazon Web Services Application Performance Management Microsoft Azure Batch Processing Cloud Computing Databases Continuous Integration DevOps Middleware Monitoring of Systems Python (Programming Language)
+15 more
Log Analysis Reliability Engineering Ansible Prometheus Datadog Google Cloud Cloud Monitoring Grafana Software Troubleshooting Kubernetes Performance Monitor Cloudwatch Splunk Appdynamics Dynatrace

Job description

We are seeking an experienced Splunk Monitoring & Observability Engineer with strong hands-on expertise in Splunk monitoring. The role will focus on reviewing and enhancing the existing monitoring landscape, optimizing Splunk usage and alerting, identifying monitoring gaps, and proposing modern observability techniques to improve proactive detection, troubleshooting, and end-to-end service visibility., * Design, develop, and enhance monitoring solutions using Splunk Enterprise/Splunk Cloud.

  • Develop and optimize SPL queries, dashboards, reports, alerts, and correlation rules.
  • Monitor applications, infrastructure, databases, APIs, middleware, and batch processes.
  • Review the current monitoring environment and identify gaps, duplicate/noisy alerts, ineffective thresholds, and performance issues.
  • Propose and implement monitoring optimization and alert rationalization initiatives.
  • Analyze incidents and production issues to support troubleshooting and root cause analysis.
  • Assess observability maturity across Logs, Metrics, Traces, and Events.
  • Propose modern observability techniques such as distributed tracing, service health monitoring, SLIs/SLOs, and proactive monitoring.
  • Enable end-to-end visibility across applications, infrastructure, cloud, and dependent services.
  • Identify opportunities for automation, monitoring-as-code, AIOps, and proactive/self-healing capabilities.
  • Collaborate with Application, Infrastructure, Cloud, SRE, DevOps, and Operations teams.

Requirements

  • Strong hands-on experience with Splunk Enterprise and/or Splunk Cloud.
  • Expertise in SPL, dashboards, alerts, reports, and log analytics.
  • Experience in monitoring optimization, alert tuning, and alert rationalization.
  • Strong troubleshooting and Root Cause Analysis (RCA) skills.
  • Understanding of observability principles: Logs, Metrics, Traces, and Events.
  • Experience monitoring applications, APIs, databases, and infrastructure.
  • Knowledge of cloud monitoring across AWS, Azure, or Google Cloud Platform.
  • Automation/scripting experience using Python, Shell, or similar technologies.

Preferred Skills

Splunk ITSI, Splunk Observability Cloud, OpenTelemetry, Grafana, Prometheus, Dynatrace,

AppDynamics, Datadog, Kubernetes, AWS CloudWatch, CI/CD, Ansible, and AIOps.

Expected Outcome The candidate should act not only as a Splunk Monitoring Engineer, but as a Monitoring & Observability SME who can: Assess Identify Gaps Optimize Rationalize Propose Observability Improvements Implement Continuous Improvements

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

12:33 min

Exploring advanced observability stacks and distributed infrastructure challenges

Pawel Piwosz · LIVE

10:40 min

Visualizing Prometheus open metrics using custom Grafana dashboards

Stijn Polfliet · LIVE

2:38 min

Establishing comprehensive monitoring and log management

Michael Eder +1 · LIVE

2:17 min

Mapping the maturity roadmap for scaled devops adoption

Dominik Krichbaum Dominik Krichbaum · World Congress 2026 Europe

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

1:04 min

Visualizing Keycloak performance via standard Grafana troubleshooting dashboards

Alexander Schwartz Alexander Schwartz · World Congress 2025

Videos

See all

Related articles

See all