> Markdown version of [/jobs/ext/330168-data-engineer-kafka](https://www.wearedevelopers.com/jobs/ext/330168-data-engineer-kafka). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer (Kafka) - **Company:** Verifone - **Location:** Ashford, UK (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Web Services, Amazon Elastic Compute Cloud, Bash Shell, Cloud Computing, Cloud Engineering, Databases, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Security, Relational Databases, Linux, DevOps, Distributed Systems, Fault Tolerance, Identity and Access Management, Python (Programming Language), Key Management, Kernel-Based Virtual Machine, PostgreSQL, Machine Learning, MongoDB, MySQL, Networking Basics, Systems Development Life Cycle, Role-Based Access Control, Redis, Ansible, Shell Script, Simple Data Format, Software Engineering, Data Streaming, Parquet, Data Processing, Scripting, Cloud Platform System, Amazon ElastiCache, Delivery Pipeline, Snowflake, Grafana, Caching, Indexer, Infrastructure as Code (IaC), Amazon Virtual Private Cloud (VPC), Data Strategy, Containerization, Data Lakes, Kubernetes, Apache Flink, Data Analytics, Apache Kafka, Verifone, Cloudwatch, Terraform, Stream Processing, Data Pipelines, Docker, Confluent, Vmware, Microservices - **Published:** June 4, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=3f65c9ab611de1fc ## About the Role Do you have experience in VMware?, * 4+ years of overall engineering experience with 2+ years of hands-on experience supporting Kafka infrastructure in a large-scale production environment. * Kafka Producer/Consumer microservices concepts and Kafka distributed architecture. * Solid Linux fundamentals: networking basics, logs, system troubleshooting, process/memory, disk. * Strong scripting and automation skills (Python, Bash) with a track record of reducing operational toil. * CI/CD pipeline development and Infrastructure-as-Code experience (Terraform preferred). * Cloud engineering skills, preferably AWS (EC2, VPC, IAM, MSK/ElastiCache, CloudWatch). * Familiarity with observability tools (metrics/logs/tracing concepts) and incident response practices. * Basic understanding of distributed systems tradeoffs (availability, consistency, partitions, backpressure). * Strong communication and presentation skills with emphasis on executive communication. * Flexible with regards to working shifts; on-call and weekends. Preferred Skills (Highly Desired) Data Engineering & Pipelines * Data engineering skills including data analytics, data processing, ETL, and data lake technologies (batch and streaming, file formats like Parquet, table formats like Iceberg/Delta/Hudi, basic orchestration). * Experience with AWS data tools (Athena, Glue, Iceberg, Redshift, etc.). * Exposure to Kafka Streams, Apache Flink, or similar stream processing frameworks. Database & Caching Technologies * Operate Redis deployments for caching, ephemeral state, queues/streams, and rate limiting use cases. * Relational DB experience: PostgreSQL and/or MySQL (indexing basics, vacuum/analyze, query plans, replication fundamentals). * MongoDB operational familiarity (replica sets, elections, oplog basics, backup/restore). Infrastructure & Security * Container/Kubernetes familiarity (deployments, stateful workloads, storage classes). * On-prem experience (VMware/KVM, storage, networking). * Security fundamentals: least privilege, secrets management, encryption-in-transit/at-rest concepts. * Experience working with PCI (Payment Card Industry Data Security) standards. ## Description Kafka Operations & Reliability * Manage and support Apache Kafka clusters (including MSK), Kafka Connect ecosystem, and KSQL for high-throughput, fault-tolerant messaging and event streaming pipelines. * Monitor system health, set up alerts, and drive incident response and root cause analysis (RCA). * Handle day-to-day operational tasks including offset management, lag monitoring, consumer group management, and cluster rebalancing. * Perform routine cluster maintenance: upgrades, configuration changes, scaling, and health checks. Scripting, Automation & DevOps * Develop and manage automation scripts using Python and Shell scripting to reduce manual toil and improve operational efficiency. * Design, implement, and maintain robust CI/CD pipelines for patching automation. * Build and maintain Infrastructure as Code (IaC) using tools such as Terraform, Ansible, or similar. * Containerize and orchestrate workloads using Docker and Kubernetes, with the opportunity to help migrate Kafka and related services to Kubernetes. Data Pipeline Development * Build and optimize end-to-end data pipelines, with Kafka as a core streaming component alongside batch and ETL processes, enabling new AI, ML, and analytics use cases. * Collaborate with development teams to implement best practices for data flow, security, scalability, and performance. * Continuously identify and implement improvements to the software development lifecycle (SDLC). Cloud & Infrastructure * Manage cloud infrastructure on AWS (VMs, networking, storage, IAM, etc.). * Support cloud migration tasks and assist in moving workloads between on-prem and cloud environments., * Direct impact on Verifone's global payment infrastructure-your work keeps billions of transactions flowing reliably every year. * Ground-floor involvement in building the data pipelines that will drive Verifone's AI, machine learning, and analytics strategy. * Hands-on participation in a Kafka-to-Kubernetes migration-a resume-building modernization project from day one. * Multi-technology exposure-Kafka, Redis, MongoDB, PostgreSQL, MySQL, Snowflake-so you grow as a well-rounded platform engineer, not a single-tool specialist. * Mentorship from senior engineers and Kafka architects with deep production experience. * Clear career progression path from mid-level to senior and lead engineer. * Funded certification paths-Confluent (CCDAK, Certified Administrator), AWS (Solutions Architect, MSK Specialty), CKA-plus budget for courses and technical conferences. * Hybrid/remote working flexibility with a collaborative team culture. ## Related Videos - [Reducing LLM Calls with Vector Search Patterns - Raphael De Lio (Redis)](https://www.wearedevelopers.com/videos/1714-reducing-llm-calls-with-vector-search-patterns-raphael-de-lio-redis) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [Inside Bitpanda's Tech Stack: Scaling a European Fintech Leader - Markus Dorner](https://www.wearedevelopers.com/videos/1979-inside-bitpanda-s-tech-stack-scaling-a-european-fintech-leader-markus-dorner) - [Accelerating Authentication Architecture: Taking Passwordless to the Next Level](https://www.wearedevelopers.com/videos/733-accelerating-authentication-architecture-taking-passwordless-to-the-next-level) - [Let's Get Started With Apache Kafka® for Python Developers](https://www.wearedevelopers.com/videos/565-let-s-get-started-with-apache-kafka-for-python-developers) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Highest Paying Tech Companies in Europe](https://www.wearedevelopers.com/magazine/162-highest-paying-tech-companies-in-europe) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)