> Markdown version of [/jobs/ext/2062156-senior-data-engineer-web-scraping](https://www.wearedevelopers.com/jobs/ext/2062156-senior-data-engineer-web-scraping). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - Web Scraping - **Company:** Jobgether - **Location:** Netherlands (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** HTML, JavaScript (Programming Language), Application Programming Interfaces (APIs), Airflow, Amazon S3, Big Data, Computer Programming, Databases, Continuous Integration, Data Cleansing, Information Engineering, Web Scraping, Data Transformation, Data Warehousing, Fiddler (Software), Github, Python (Programming Language), Standard Sql, Selenium, Workflow Management Systems, XPath, Data Processing, Postman, Pandas, Kubernetes, Information Technology, Web Technologies, Data Delivery, Amazon Simple Queue Service (SQS), Data Pipelines, Docker, Jenkins - **Published:** August 15, 2026 - **Apply:** https://www.adzuna.nl/details/5843238600 ## About the Role * Bachelor's or master's degree in Computer Science, Engineering, or a related technical discipline. * 4-6 years of professional experience in data engineering or a closely related field. * Strong programming skills in Python and strong knowledge of SQL and database technologies. * Advanced hands-on expertise with the Python Pandas library for data cleaning, manipulation, exploration, and transformation. * Strong web-scraping experience with tools and technologies such as Selenium, Scrapy, Fiddler, Postman, and XPath. * Strong experience with Apache Airflow for workflow orchestration and data pipeline management. * Solid understanding of web technologies, including HTML, JavaScript, APIs, and related concepts. * Proven experience working with large datasets and performing data cleaning, transformation, manipulation, and replacement. * Ability to design scalable infrastructure, data products, and technical tools for data-focused teams. * Strong verbal and written communication skills, with the ability to collaborate effectively with technical and non-technical stakeholders. * Self-motivated, detail-oriented, and comfortable working independently while taking ownership of projects and outcomes. * Experience with Docker and workload containerization is preferred; Kubernetes experience is a plus. * Familiarity with automation and CI/CD technologies such as Jenkins and GitHub Actions is an advantage. * Experience with AWS services such as S3, RDS, SNS, SQS, and Lambda is a plus. * Fully remote position with the flexibility to work from anywhere. * Full-time opportunity within a collaborative, team-oriented engineering environment. * Significant autonomy, ownership, and trust in how you approach technical challenges. * Opportunity to work on sophisticated web-scraping, data engineering, automation, and data-product initiatives. ## Description This is a fully remote opportunity for a data engineering professional specializing in web scraping, data processing, and automation. You'll design and maintain sophisticated scrapers that transform diverse web-based sources into reliable, high-quality datasets. Your work will directly support analytical and investment-related decisions by delivering timely data, alerts, and production-ready data products. The role combines hands-on Python development, data transformation, database engineering, and workflow orchestration. You'll collaborate closely with analysts, engineers, and cross-functional teams to understand requirements and build scalable solutions. With significant ownership and autonomy, you'll have the opportunity to improve platforms, automate processes, and solve challenging data problems. The environment is entrepreneurial and team-oriented, with a strong focus on engineering quality, operational reliability, and continuous innovation. Accountabilities: * Design, develop, deploy, and maintain web scrapers using a range of scraping techniques and tools to collect alternative datasets from diverse sources. * Use Python and Pandas to clean, explore, transform, manipulate, and prepare large datasets for downstream consumption. * Build and maintain efficient data pipelines that ingest scraped data into databases and data warehouses. * Develop and manage scheduled workflows using Apache Airflow and other orchestration tools to ensure reliable and timely data delivery. * Collaborate with analysts and cross-functional stakeholders to understand current and anticipated data requirements and translate them into effective technical solutions. * Develop quality-control checks to validate data availability, accuracy, consistency, and integrity. * Maintain alerting systems, investigate time-sensitive data incidents, and resolve operational issues to ensure reliable day-to-day data delivery. * Design and implement tools, applications, and automation that improve the capabilities and efficiency of the web-scraping platform. * Contribute to infrastructure and data-product design, bringing practical solutions that support data scientists and other technology teams. * Work independently while collaborating with engineering, product, and technology stakeholders to deliver high-quality solutions and continuously improve existing systems., * Exposure to complex datasets supporting analytical and investment-related decision-making. * Collaboration with engineering, product, analysts, data scientists, and other technology professionals. * Opportunity to contribute to the development and evolution of an entrepreneurial technology team. * Professional environment focused on innovation, continuous improvement, and operational excellence. ## Related Videos - [Scrape, Train, Predict: The Lifecycle of Data for AI Applications](https://www.wearedevelopers.com/videos/1652-scrape-train-predict-the-lifecycle-of-data-for-ai-applications) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [The Resilience of the World Wide Web](https://www.wearedevelopers.com/videos/1281-the-resilience-of-the-world-wide-web) - [Modern Web Data Extraction: Techniques, Tools, and Legal & Ethical Considerations](https://www.wearedevelopers.com/videos/2046-modern-web-data-extraction-techniques-tools-and-legal-ethical-considerations) - [From clicks to cribs - How to find your dream home with web scraping](https://www.wearedevelopers.com/videos/767-from-clicks-to-cribs-how-to-find-your-dream-home-with-web-scraping) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) ## Related Articles - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs) - [Find a Developer Job: 12 Best Job Sites For Developers](https://www.wearedevelopers.com/magazine/165-find-a-developer-job-12-best-job-sites-for-developers) - [Software Developer Salary in The Netherlands [2023]](https://www.wearedevelopers.com/magazine/217-software-developer-salary-in-the-netherlands-2023) - [Where To Find Software Engineering Jobs](https://www.wearedevelopers.com/magazine/396-where-to-find-software-engineering-jobs) - [How to Find Tech Jobs in Amsterdam](https://www.wearedevelopers.com/magazine/279-how-to-find-tech-jobs-in-amsterdam) - [Where to Find Entry-Level Software Engineering Jobs](https://www.wearedevelopers.com/magazine/397-where-to-find-entry-level-software-engineering-jobs)