> Markdown version of [/videos/2114-easy-mode-monitoring-and-logging-with-shiftmon?t=0](https://www.wearedevelopers.com/videos/2114-easy-mode-monitoring-and-logging-with-shiftmon?t=0). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Easy Mode Monitoring and Logging with Shiftmon

Ditch expensive SaaS observability tools and heavy infrastructure. ShiftMon uses Ansible to automate a production-ready metrics and logging stack on a single Linux VM.

- **Speakers:** [Mathias Palmersheim](https://www.wearedevelopers.com/@mathias-palmersheim)
- **Event:** World Congress 2026 Europe - Virtual Stage
- **Published:** July 3, 2026
- **Duration:** 32:00
- **URL:** https://www.wearedevelopers.com/videos/2114-easy-mode-monitoring-and-logging-with-shiftmon

## Summary

ShiftMon serves as an open-source observability on-ramp designed to eliminate the heavy infrastructure footprint and expensive SaaS subscriptions typically associated with platform monitoring. Glued together with Ansible, it automates the deployment of a complete metrics, logs, and traces stack on a small Linux virtual machine. By combining Traefik for TLS automation, Grafana for visualization, VM Auth for request authorization, and the VictoriaMetrics suite for data storage, ShiftMon provides a production-ready baseline that remains easily extensible. The platform emphasizes intelligent data collection using Telegraf, natively supporting structural enumeration for common services like Docker while relying on an automated baseline for essential host metrics. A significant design consideration is the strategic choice between push and pull telemetry. While service discovery and metric scraping excel in Kubernetes or static cloud environments, push-based pipelines are heavily recommended for ephemeral workflows like CI runners or mobile laptops. Additionally, black-box monitoring stands out as a critical methodology for tracking raw availability and latency without requiring internal application knowledge, leveraging tools like VM Anomaly to detect subjective performance degradation. Building a resilient observability culture ultimately means planning for failure and prioritizing developer experience. Infrastructure configuration should always be managed as code, allowing entire monitoring environments to be restored from scratch in under an hour if hardware fails. When configuring metric collectors, explicitly avoid running them as root; their ability to execute arbitrary commands creates a dangerous vulnerability pivot point if compromised. Similarly, teams should favor data source-managed alerts over application-managed UI alerts to reduce architectural complexity and ensure heavy user queries do not block critical notifications. Alert thresholds themselves must be written with high empathy for the on-call engineer, specifically designing for '2:00 a.m. you' rather than an idealized daytime scenario. Finally, engineers can maintain dashboard portability by eliminating hard-coded data source IDs, actively preventing vendor lock-in during future architectural migrations.

**Keywords:** open source observability stack, Ansible infrastructure as code, VictoriaMetrics data storage, Traefik TLS automation, Telegraf metrics collection, push vs pull telemetry, black box monitoring availability, data source managed alerts, least privilege monitoring agents, observability pipeline configuration, Grafana dashboard portability, ephemeral CI worker monitoring, Docker structured logging, OpenTelemetry ingress routing, ShiftMon deployment strategy

## Chapters

1. **Introduction to easy mode observability and ShiftMon** (00:00) — Automating difficult infrastructure setups via open-source tools resolves the underlying friction of adopting advanced observability practices.
1. **Setting up observability without expensive SaaS subscriptions** (00:45) — Deploying a small Linux virtual machine provides an easily extensible observability on-ramp bypassing costly subscriptions.
1. **Deploying configuration as code seamlessly using Ansible** (02:05) — Converting a plain Linux machine into an observability stack via infrastructure as code eliminates the need for managed configuration agents.
1. **Connecting core components to build observability platforms** (02:52) — Integrating reverse proxies and dedicated data storage systems constructs a robust monitoring backend environment.
1. **Automating telemetry collection through robust Telegraf deployment** (06:09) — Using Ansible roles to discover background processes automatically configures the correct host metric collection patterns.
1. **Streamlining baseline observability setup for lazy engineers** (06:55) — Automating host statistics and standard application instrumentation eliminates the manual burden of configuring baseline telemetry.
1. **Discovering and instrumenting services using systemd process enumeration** (08:30) — Detecting local services automatically to scrape unauthenticated endpoints streamlines collection while allowing custom user overrides.
1. **Deciding between push metric architecture and service discovery** (11:01) — Why ephemeral workloads and highly mobile nodes necessitate push-based monitoring instead of traditional scrape architectures.
1. **Instrumenting supported infrastructure components for automated telemetry scraping** (13:53) — Automating metric ingestion across supported infrastructure components enables deep visibility into networking domains and container hosts.
1. **Measuring application availability thresholds using black box monitoring** (17:03) — Deploying external HTTP probes and anomaly detection logic identifies unexpected performance latency across digital services.
1. **Managing tokens and user credentials securely across endpoints** (18:25) — Storing sensitive observability parameters inside external secret managers avoids exposing plaintext configurations on host deployments.
1. **Customizing dashboards and alert rules for proprietary applications** (19:31) — Creating tailored observability rule sets and collector configurations preserves the ability to monitor unsupported internal services.
1. **Securing telemetry monitoring agents to prevent privilege escalation** (21:51) — Running telemetry collection tools strictly as unprivileged users prevents arbitrary command execution and security vulnerabilities.
1. **Provisioning dashboards and data source alerts as code** (24:04) — Maintaining visualization definitions and stateless alerting rules as versioned code enables rapid infrastructure disaster recovery.
1. **Exploring observability dashboards and alert management interface workflows** (28:01) — Touring a live user interface reveals how authenticated operators investigate log correlation traces and monitor fleet performance.

## Related Moments

- [Background and origins of the Shiftmon monitoring project](https://www.wearedevelopers.com/videos/2118-better-together-leveraging-your-observability-tools-as-a-siem) (from "Better Together: Leveraging Your Observability Tools as a SIEM")
- [Exploring advanced observability stacks and distributed infrastructure challenges](https://www.wearedevelopers.com/videos/544-plan-ci-cd-on-the-enterprise-level) (from "Plan CI/CD on the Enterprise level!")
- [Introducing Monoscope for intelligent post-deployment system monitoring](https://www.wearedevelopers.com/videos/1639-apitoolkit-using-merkle-trees-and-llms-to-detect-the-undetectable-in-software-monitoring) (from "APItoolkit: Using Merkle Trees and LLMs to Detect the UnDetectable in Software Monitoring")
- [Building a unified stack leveraging core open source tools](https://www.wearedevelopers.com/videos/2118-better-together-leveraging-your-observability-tools-as-a-siem) (from "Better Together: Leveraging Your Observability Tools as a SIEM")
- [Introduction to metrics and observability challenges in monitoring](https://www.wearedevelopers.com/videos/57-all-your-telemetry-data-from-any-source-in-one-place) (from "All your telemetry data from any source in one place")
- [Implementing a robust observability stack for agents](https://www.wearedevelopers.com/videos/100269-beyond-the-benchmark-how-to-evaluate-ai-agents-in-the-real-world) (from "Beyond the Benchmark: How to Evaluate AI Agents in the Real World")

## Related Articles

- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [Dev Digest 131 - AI'm not sure about OSS](https://www.wearedevelopers.com/magazine/472-dev-digest-131-ai-m-not-sure-about-oss)
- [Effortlessly Scale Prometheus With The Telemetry Data Platform – And Keep your Grafana Dashboards, Too!](https://www.wearedevelopers.com/magazine/3-effortlessly-scale-prometheus-with-the-telemetry-data-platform-and-keep-your-grafana-dashboards-too)
- [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)

## Related Jobs

- [Software Engineer, Platform Engineering (L2)](https://www.wearedevelopers.com/jobs/ext/1956829-software-engineer-platform-engineering-l2) at **Twilio**
- [Senior Engineer, Infrastructure Platform](https://www.wearedevelopers.com/jobs/ext/328836-senior-engineer-infrastructure-platform) at **Intercom, Inc.**
- [Software Engineer L2 - Cloud Infrastructure](https://www.wearedevelopers.com/jobs/ext/1282024-software-engineer-l2-cloud-infrastructure) at **Twilio**
- [Software Engineer L2 - Cloud Infrastructure](https://www.wearedevelopers.com/jobs/ext/1293339-software-engineer-l2-cloud-infrastructure) at **Twilio**
- [Software Engineer L2 - Cloud Infrastructure](https://www.wearedevelopers.com/jobs/ext/1644423-software-engineer-l2-cloud-infrastructure) at **Twilio**
- [Engineer, Offensive Security Organization](https://www.wearedevelopers.com/jobs/ext/1992296-engineer-offensive-security-organization) at **Twilio**