Data Architect - AI & Cloud Infrastructure Leeds Halifax Manchester

We are seeking a visionary Lead Data Architect to spearhead the evolution of our enterprise data platform. In this role, you will bridge the gap between traditional data engineering and cutting-edge artificial intelligence. You will not build AI models from scratch; instead, you will architect the scalable frameworks, high-performance pipelines, and secure storage systems that power our Generative AI (GenAI) and Predictive Machine Learning (ML) initiatives. The ideal candidate will design enterprise-grade Blueprints for Vector databases, RAG (Retrieval-Augmented Generation) infrastructure, and unified data lakes that ensure our AI assets are secure, governed, and highly available., 1. AI & Generative AI Infrastructure Design

• Architect RAG Pipelines: Design scalable end-to-end Retrieval-Augmented Generation infrastructure to inject real-time enterprise context into Large Language Models (LLMs).
• Vector Storage Management: Select, implement, and optimise enterprise vector databases (e.g., Pinecone, Milvus, pgvector) for high-performance embedding storage and semantic search.
• Agentic AI Enablement: Build high-throughput, low-latency data loops required to support autonomous AI agents in production.

2. Core Data Architecture & MLOps Integration

• Unified Data Foundations: Scale our modern data stack utilizing Lakehouse architectures (e.g., Delta Lake, Apache Iceberg) to handle both unstructured AI data and structured analytics.
• Feature Engineering Infrastructure: Design and maintain enterprise Feature Stores (e.g., Feast, Tecton) to serve unified data features consistently across offline training and online real-time inference.
• Streamline MLOps Pipelines: Partner with ML Engineers to integrate data pipelines seamlessly with lifecycle tracking frameworks like MLflow or Kubeflow.

3. AI Data Governance, Privacy & Quality

• Data Lineage Automation: Implement comprehensive data lineage tracking to audit the source datasets used for AI training, fine-tuning, and prompt context.
• Security & Compliance: Architect automated data masking, anonymisation, and PII-filtering pipelines to prevent sensitive data from leaking into foundational models.
• AI Data Cataloguing: Curate metadata structures within platform catalogs (e.g., Collibra, Atlan) to explicitly map physical data assets to corresponding AI applications.

• Experience: Minimum of 7+ years of experience in data architecture, data engineering, or enterprise infrastructure design.
• Cloud Mastery: Deep architectural expertise in at least one major cloud platform and its AI ecosystem:

• AWS: SageMaker, Bedrock, Glue, Redshift.
• Azure: Azure OpenAI Service, Azure Machine Learning, Synapse.
• GCP: Vertex AI, BigQuery ML, Dataflow.

• Advanced Data Modeling: Proven success modeling for both traditional relational/NoSQL analytical engines and high-dimensional vector spaces.
• Data Pipeline Frameworks: Hands-on experience with streaming and batch tooling including Apache Spark, Kafka, Flink, and orchestration tools like Apache Airflow or Prefect.
• Programming Literacy: Strong proficiency in Python, SQL, and database internals.

Preferred Qualifications

• Experience with unified data clouds such as Databricks or Snowflake.
• Relevant cloud certifications (e.g., AWS Certified Data Engineer, Azure Solutions Architect, Google Cloud Professional Data Engineer).
• Familiarity with Docker, Kubernetes, and infrastructure-as-code (Terraform).