> Markdown version of [/jobs/ext/347968-data-engineer](https://www.wearedevelopers.com/jobs/ext/347968-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** Obsidian Security
- **Location:** Manchester, UK (Remote available)
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Amazon S3, Business Analytics Applications, Cloud Database, Code Review, Information Engineering, Data Transformation, Data Stores, Relational Databases, Python (Programming Language), PostgreSQL, Operational Databases, Prometheus, Standard Sql, Software Engineering, Datadog, Large Language Models, Grafana, Apache Spark, Backend, Git, Containerization, Gitlab-ci, Kubernetes, Apache Kafka, Stream Processing, Data Pipelines, Docker, Databricks
- **Published:** June 20, 2026
- **Apply:** https://uk.indeed.com/viewjob?jk=6c78776f85723cc7

## About the Role

Do you have experience in Spark?, Do you have a Master's degree?, * 3-6 years of experience in a data engineering or software engineering role

 * Strong SQL and hands-on experience building production data pipelines
 * Experience with a modern data orchestrator such as Dagster, Airflow, or similar
 * Proficiency with a data transformation framework such as dbt
 * Proficiency in Python
 * Familiarity with Git and CI/CD tooling such as GitLab CI/CD
 * Familiarity with relational databases (e.g., Postgres) and cloud data warehouses
 * A bias toward data quality, testing, and maintainable, well-documented work
 * Experience collaborating in a team environment and adapting to changing requirements, * Experience with Databricks/Spark or other large-scale analytics platforms
 * Exposure to event/streaming systems such as Kafka
 * Existing Go experience for occasional API changes
 * Familiarity with containerization and orchestration (Docker, Kubernetes)
 * Exposure to LLM-assisted data workflows
 * Experience with cloud platforms (AWS or GCP) and object storage (S3/GCS)
 * Exposure to observability tooling such as Grafana or Prometheus

## Description

As a Data Engineer at Obsidian, you'll:

 * Own and evolve the data pipelines behind one of Obsidian's core products
 * Build and maintain data transformations and orchestrated jobs that turn raw signal into attributed, enriched data at scale
 * Keep customer analytics datasets accurate, fresh, and reliable across multiple data stores
 * Extend and improve our data attribution engine, including its rule-based and LLM-assisted curation stages
 * Make occasional changes to the Go service that exposes this data through our APIs, working alongside backend engineers
 * Collaborate with product managers, backend engineers, and the teams who turn this data into customer-facing functionality
 * Champion data correctness, observability, and clean, testable transformations
 * Participate in code reviews and technical discussions to continuously raise engineering standards


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)
- [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)