> Markdown version of [/jobs/ext/4141235-senior-data-ai-engineer](https://www.wearedevelopers.com/jobs/ext/4141235-senior-data-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data & AI Engineer

- **Company:** BASF SE
- **Location:** Madrid, Spain (Remote available)
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Microsoft Azure, Big Data, Cloud Computing, Cloud Storage, Continuous Integration, Data Mining, DevOps, Elasticsearch, Information Extraction, Python (Programming Language), Knowledge Management, Natural Language Processing, Performance Tuning, Scrum Methodology, Search Technologies, Software Engineering, Enterprise Search, Cloud Platform System, Azure Data Factory, Retrieval-Augmented Generation, Large Language Models, Apache Spark, Pyspark, Kubernetes, Information Technology, OpenSearch, Databricks
- **Published:** October 31, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=9baf509aa3a0204d

## About the Role

* A degree in computer science, software engineering, mathematics or similar experience
 * Strong problem-solving skills with the ability to break down complex data and system challenges into pragmatic, scalable solutions
 * Ownership mindset: takes responsibility end-to-end, from design and implementation to production support and continuous improvement
 * Experience building and operating production-grade big data pipelines on Databricks
 * Experience applying AI/ML and NLP techniques for information extraction, retrieval or knowledge discovery
 * Experience with modern LLM-based RAG architectures, embeddings and vector search
 * Hands-on expertise with Apache Spark (PySpark/Scala), including performance tuning, data skew handling, and efficient cluster usage
 * Solid Python programming skills
 * Experience with ElasticSearch / OpenSearch
 * Experience working in cloud-based DevOps environments, including Azure cloud, Azure DevOps, and CI/CD pipelines
 * Fluent communication in English is required

Nice to have

 * Experience working with Azure Data Factory (ADF)
 * Experience working with Kubernetes
 * Experience in cloud environments (Storage Accounts, ADLS2)
 * Experience with multimodal document processing (text, images, scanned documents)
 * Experience evaluating and benchmarking LLMs for enterprise search and knowledge management

## Description

* Building data and AI pipelines for document processing in our (micro-)batch architecture
 * Collaborating with subject matter experts to understand requirements
 * Designing and implementing data extraction and retrieval solutions, leveraging modern AI methods where appropriate
 * Introducing new technologies to deliver reliable and cost-efficient services
 * Contributing to the evolution of our AI-powered search and knowledge platforms, including multimodal retrieval and knowledge extraction capabilities
 * Working in an international (team language English) and agile team (Scrum)