> Markdown version of [/jobs/ext/3305372-senior-data-engineer-terminal-exe](https://www.wearedevelopers.com/jobs/ext/3305372-senior-data-engineer-terminal-exe). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer - Terminal.exe

- **Company:** All Your BI
- **Location:** London, UK (Remote available)
- **Experience:** Expert
- **Salary:** £85,000.0 - £99,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Microsoft Azure, Software as a Service, Cloud Computing, Databases, Data Architecture, Information Engineering, Data Mart, Python (Programming Language), PostgreSQL, Microsoft SQL Server, Octopus Deploy, Operational Data Store, Operational Databases, Azure DevOps Pipelines, Standard Sql, Data Streaming, YAML, Cloud Platform System, Apache Kafka, Data Pipelines
- **Published:** September 9, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=312c3c27ae58e4d6

## About the Role

* Attitude and experience: curiosity and a genuine appetite for architectural ownership, and 5+ years of data engineering experience, including end-to-end ownership of production data platforms. This is a product under active development, we need you comfortable deciding, not just executing.
 * Core stack: strong SQL and Python; Go experience is a significant plus (our connectors and consumers are written in Go). Solid experience with SQL Server as a source system and PostgreSQL as a target store.
 * Pipelines and streaming: experience building and operating ELT pipelines against operational source systems, including watermark/incremental or CDC/replica-based extraction, and with event-driven or streaming architectures, we use Azure Event Hubs with a Kafka-compatible endpoint.
 * Cloud and observability: cloud data platform experience on Azure, familiarity with AKS, Azure DevOps CI/CD, and GitOps (Argo CD) is a plus, alongside a strong focus on data correctness and instrumentation, we use OpenTelemetry.
 * Nice to have: knowledge of the container terminal or port logistics industry, direct experience with Navis N4 or another TOS, multi-tenant SaaS data architecture experience, and familiarity with declarative, YAML-driven transformation frameworks. Industry knowledge is a strong plus, not a hard requirement, we'll teach you the terminal domain.

## Description

You will design and build the pipelines that move data from terminal operating systems (such as Navis N4) into our platform, and the transformation layer that turns raw operational data into curated, documented datasets.

Concretely, you will work on:

 * ELT pipelines from on-premises terminal databases into our Azure-based platform, with near-real-time refresh (3-minute microbatch) for operational reporting and daily batch for analytics. Extraction uses watermark-based incremental queries over Kafka (Azure Event Hubs) to PostgreSQL staging. A multi-tenant data architecture on Azure PostgreSQL with database-per-tenant isolation and a standardized STG ODS
 * DataMart layer structure driven by our internal declarative transformation framework (dewi)
 * Our internal connector tooling written in Go, which you will help develop and extend
 * Data quality monitoring: gap detection, freshness alerting, and reconciliation
 * The standardized terminal data model: mapping source data from different TOS versions and configurations onto shared definitions of moves, calls, and equipment events

You will work directly with terminal operations staff, without intermediate layers.


## Related Videos

- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [CI/CD with Github Actions](https://www.wearedevelopers.com/videos/856-ci-cd-with-github-actions)
- [Kubernetes and Microservices with Multi-Model Databases](https://www.wearedevelopers.com/videos/382-kubernetes-and-microservices-with-multi-model-databases)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Fault Tolerance and Consistency at Scale: Harnessing the Power of Distributed SQL Databases](https://www.wearedevelopers.com/videos/1146-fault-tolerance-and-consistency-at-scale-harnessing-the-power-of-distributed-sql-databases)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)