Lead Data Engineer - Python AI Vector PostgresSQL

MLR Associates

Charing Cross, United Kingdom

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Languages

English

Experience level

Senior

Compensation

£ 130K

Job location

Charing Cross, United Kingdom

Tech stack

Artificial Intelligence

Airflow

Software as a Service

Computer Programming

Information Engineering

Data Governance

Data Integrity

Relational Databases

Distributed Data Store

Elasticsearch

Python

PostgreSQL

MySQL

NoSQL

Product Management

Search Technologies

Software Engineering

Spark

Storage Technologies

Real Time Data

Kafka

Data Pipelines

Job description

Lead Data Engineer - Python AI Vector PostgresSQL
Leading Technology AI Brand
SaaS - Platform based Technology Services
London/City
£70-130k salary + equity package

Our client a global technology leader is currently looking for a Senior/Lead Data Engineer to work with the dev team to guide the provision of Software Development for an exciting new AI product.

Key Responsibilities:-

Architect and build scalable data pipelines and infrastructure
Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads.
Develop and manage batch and Real Time data pipelines.
Build and optimize systems for vector search, retrieval, and ML data pipelines.
Ensure data reliability, security, and governance across the platform.
Collaborate with AI and Back End engineering teams to support training, inference, and product features.
Implement monitoring, observability, and data quality frameworks.

Core Experience:-

7+ years of experience in data engineering or Back End engineering roles.
Strong experience designing and building data pipelines and distributed data systems.
Experience working with relational databases (PostgreSQL preferred, but MySQL or similar is acceptable).
Experience with NoSQL databases.
Experience with vector databases used in modern AI systems.
Strong programming experience in Python.

Frameworks/Infrastructure:-

Apache Spark
Apache Airflow
Kafka
Elasticsearch/OpenSearch

Requirements

7+ years of experience in data engineering or Back End engineering roles.
Strong experience designing and building data pipelines and distributed data systems.
Experience working with relational databases (PostgreSQL preferred, but MySQL or similar is acceptable).
Experience with NoSQL databases.
Experience with vector databases used in modern AI systems.
Strong programming experience in Python.

Frameworks/Infrastructure:-