> Markdown version of [/jobs/ext/839349-senior-software-engineer-data-pipelines](https://www.wearedevelopers.com/jobs/ext/839349-senior-software-engineer-data-pipelines). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Software Engineer - Data Pipelines - **Company:** Obsidian Security - **Location:** Cheltenham, UK (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Amazon S3, Data Architecture, Data Infrastructure, Relational Databases, Software Design Patterns, Programming Tools, Distributed Systems, Monitoring of Systems, Python (Programming Language), PostgreSQL, Machine Learning, Performance Tuning, Redis, Prometheus, Standard Sql, Software Deployment, Software Engineering, Systems Integration, Multithreading, Large Language Models, Grafana, Backend, Git, Gitlab-ci, Solid Principles, Apache Kafka, Stream Processing, Data Pipelines, Docker, Programming Languages - **Published:** June 18, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=8830b92797f83839 ## About the Role Do you have experience in Software deployment?, * 5-7 years of experience in a software engineering role * Proficiency in one or more modern programming languages such as Python, Go, or SQL * Experience building backend services, APIs, and multithreaded applications * Familiarity with containerization and orchestration technologies such as Docker and Kubernetes * Strong knowledge of relational databases (e.g., Postgres) * Experience collaborating in team environments and adapting to changing requirements * Understanding of software design principles and engineering best practices * Experience with cloud platforms (AWS, GCP), object storage (S3), or event/streaming systems (Kafka, Redis) * Familiarity with Git for version control and deployment tooling, such as GitLab CI/CD * Experience in influencing design patterns based on best practices. * Experience assessing alternative data architecture designs to produce accurate and unbiased recommendations. Desirable Experience * Experience with system monitoring and observability tools such as Grafana, Prometheus, or similar platforms * Understanding of quality engineering (QE) practices across development and testing lifecycles * Exposure to large-scale distributed systems and performance optimisation * Experience in forming strong stakeholder relationships to help promote joint working between teams. * Experience of working alongside both UK and US development teams. AI Skills & AI-Native Engineering Expectations: As an AI-forward engineering organization, we expect senior engineers to effectively leverage AI tools and understand foundational AI concepts to enhance development efficiency and build AI-ready systems. AI Engineering Capabilities: * Leverage AI tools effectively to improve development efficiency and build AI-ready systems. * Proficient with AI-powered developer tools; able to critically evaluate and refine AI-generated outputs. * Strong understanding of core AI/ML concepts (LLMs, embeddings, vector databases, inference, evaluation). * Experience integrating AI/ML APIs and building AI-ready data infrastructure (e.g., for RAG). * Ensure data quality, governance, observability, reliability, security, and performance in AI-driven systems. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Reducing LLM Calls with Vector Search Patterns - Raphael De Lio (Redis)](https://www.wearedevelopers.com/videos/1714-reducing-llm-calls-with-vector-search-patterns-raphael-de-lio-redis) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [The AI-Ready Stack: Rethinking the Engineering Org of the Future](https://www.wearedevelopers.com/videos/1706-the-ai-ready-stack-rethinking-the-engineering-org-of-the-future) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated) - [What is Software Engineering in the Age of AI?](https://www.wearedevelopers.com/magazine/640-what-is-software-engineering-in-the-age-of-ai)