> Markdown version of [/jobs/ext/910215-ai-data-engineer](https://www.wearedevelopers.com/jobs/ext/910215-ai-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Data Engineer - **Company:** Pst - **Location:** Madrid, Spain - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Cascading Style Sheets (CSS), Continuous Integration, Data Validation, Web Scraping, Data Mining, DevOps, Document Object Model, JSON, Python (Programming Language), NoSQL, Open Source Technology, SQL Databases, XPath, Large Language Models, Data Lakes, Information Technology, Playwright, Docker - **Published:** June 30, 2026 - **Apply:** https://es.trabajosdiarios.com/trabajo/3058246/ai-data-engineer-en-barcelona ## About the Role * Bachelor's degree in Computer Science * 3+ years of experience in web scraping or data extraction Required Skills: * Proficiency with Python * Experience with specification-Driven Extraction * Experience with LangChain, LangGraph, LlamaIndex, AutoGen * Hands on use of Scrapy LLM, Scrapy MCP Server, or similar systems that decouple field definitions from page structure * Familiarity with frameworks that give LLMs browser control (Playwright + MCP, BMAD/TEA) to handle complex, non deterministic crawling tasks. * Classical Scraping Fundamentals * Data Validation & Storage - Ability to define validation rules within specifications and land clean data into SQL/NoSQL databases or data lake * Basic API integration and authentication flows. * HTTP, DOM, XPath, CSS. Nice to Haves: * Contributions to open-source scraping or AI-automation projects. * Contributions to open-source scraping or AI-automation projects. * Familiarity with data privacy engineering (GDPR, CCPA) baked into specification design. * DevOps light - Docker, CI/CD for testing extraction specifications. ## Description Specification-Driven Extraction Engineering: * Design and maintain declarative extraction specifications-using Pydantic models, JSON schemas, or domain-specific languages-that describe exactly which fields to capture, their types, and validation rules. * Implement pipelines that translate these specifications into executable extraction plans, leveraging both classical (Scrapy, Playwright) and AI-augmented (LLM-based semantic parsing) backends. * Build reusable specification libraries for recurring data types (product prices, tariff codes, regulatory texts) to accelerate onboarding of new sources. * Design and implement autonomous data extraction agents that can make decisions about source selection, retry logic, and parsing strategies Autonomous & Self-Healing Systems: * Deploy self-healing spiders that automatically detect website layout changes and repair themselves using Model Context Protocol (MCP) servers (e.g., Scrapy MCP Server, Playwright MCP). * Integrate semantic extraction (Scrapy-LLM, custom LLM pipelines) to eliminate selector brittleness-spiders rely on field descriptions, not fragile XPaths. * Hands-on experience building AI agents and orchestration systems. * Orchestrate complex, multi-step browsing workflows with agentic frameworks (BMAD/TEA, AutoGPT-like agents) that reason about page state, adapt to anti-bot measures, and correct their own behaviour in real time. Platform Thinking & Reusability: * Move beyond one-off scrapers: build a component-based extraction platform where selectors, login handlers, and pagination logic are shared, versioned, and tested. * Implement monitoring, alerting, and automatic rollback for failed extraction runs. * Champion ethical crawling by design-rate limiting, robots.txt respect, and compliance with GDPR/CCPA are built into the specification layer, not retrofitted. Collaboration & Continuous Innovation: * Partner with data scientists and domain experts to refine extraction specifications for complex, unstructured domains (e.g., legal texts, tariff classifications). * Evaluate and pilot emerging tools to push automation coverage beyond 90%. * Document and evangelise specification-driven best practices across the engineering organisation. ## Related Videos - [Scrape, Train, Predict: The Lifecycle of Data for AI Applications](https://www.wearedevelopers.com/videos/1652-scrape-train-predict-the-lifecycle-of-data-for-ai-applications) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [Are Classical Automation Frameworks Dead? How AI Agents Are Transforming QA](https://www.wearedevelopers.com/videos/100243-are-classical-automation-frameworks-dead-how-ai-agents-are-transforming-qa) - [NoSQL Data Modeling for Front-end Developers](https://www.wearedevelopers.com/videos/297-nosql-data-modeling-for-front-end-developers) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)