> Markdown version of [/jobs/ext/4033049-python-developer-web-scraping-data-extraction](https://www.wearedevelopers.com/jobs/ext/4033049-python-developer-web-scraping-data-extraction). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Python Developer (Web Scraping & Data Extraction) - **Company:** QUALITEST - **Location:** Washington, DC, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** JavaScript (Programming Language), Application Programming Interfaces (APIs), Artificial Intelligence, Airflow, Amazon Web Services, Data Analysis, Automation of Tests, Microsoft Azure, BigQuery, Cascading Style Sheets (CSS), Cloud Storage, Code Review, Databases, Concurrent Computing, Continuous Integration, Information Engineering, Data Governance, Web Scraping, Data Mining, Data Warehousing, Software Debugging, Web Development, Web Browsers, Python (Programming Language), PostgreSQL, MongoDB, Multiprocessing, NoSQL, NumPy, Reference Data, Regular Expressions, Standard Sql, Selenium, Software Engineering, XPath, Data Processing, Multithreading, Scripting, Cloud Platform System, Snowflake, Apache Spark, Puppeteer (Software), Git, Pandas, Pytest, Kubernetes, Information Technology, Playwright, Apache Kafka, Non-relational Database, Graphql, Data Management, Dynamic Content, Data Pipelines, Docker, Amazon Redshift - **Published:** October 30, 2026 - **Apply:** https://www.careerbuilder.com/job-details/python-developer-web-scraping-data-extraction-washington-dc--a3663959-1249-4f8c-8703-3fd279d5cb2b ## About the Role Python Programming/Scripting Language, Web Programming, Quality Engineering, Artificial Intelligence (AI), Python Programming/Scripting Language, Web Programming, Software Engineering, Application Programming Interface (API), Data Collection, Data Analysis, * 5-8 years of professional Python development experience, including at least 3 years focused on web scraping or data extraction * Strong hands-on experience with Scrapy, BeautifulSoup, Requests/HTTPX, and Selenium, Playwright, or Puppeteer * Experience with asynchronous and concurrent programming (asyncio, multithreading, multiprocessing) * Solid understanding of HTTP, HTML/CSS, DOM structure, XPath/CSS selectors, cookies, sessions, and authentication flows * Experience with proxies, user-agent rotation, headless browsers, and techniques for avoiding detection * Proficiency in data processing with Pandas, NumPy, and regular expressions * Working knowledge of SQL and at least one NoSQL database (e.g., PostgreSQL, MongoDB) * Experience building and scheduling data pipelines (e.g., Airflow, Prefect, or cron-based orchestration) * Familiarity with Git, CI/CD, Docker, and automated testing (pytest) * Experience with a cloud platform (AWS, Azure, or GCP) * Strong debugging and problem-solving skills, and clear written and verbal communication * Bachelors degree in Computer Science, Engineering, Mathematics, Finance, or a related field (or equivalent experience) Preferred Qualifications (Finance / Fintech / Investments) * Background in financial services, fintech, asset management, or investment research * Experience collecting and structuring data such as market prices, company filings (SEC/EDGAR), earnings reports, news, ESG data, alternative data, or fund and security reference data * Familiarity with financial concepts such as equities, fixed income, funds, portfolios, and risk and return metrics * Understanding of data quality, lineage, and audit requirements in regulated environments * Awareness of financial data licensing, compliance, and data governance considerations * Experience with financial data providers and APIs (e.g., Bloomberg, Refinitiv, Alpha Vantage, Yahoo Finance, Polygon) * Knowledge of NLP or text extraction from PDFs, filings, and unstructured documents * Experience with big data tools (Spark, Kafka) or data warehouses (Snowflake, BigQuery, Redshift) * Exposure to Kubernetes, infrastructure as code, or distributed scraping architectures Key Competencies * Reliability mindset: builds scrapers that keep running and recover gracefully when sites change * Data quality focus: treats accuracy and completeness as non-negotiable * Ownership: takes responsibility for solutions end to end, from design to monitoring * Collaboration: communicates clearly with technical and non-technical stakeholders * Ethical judgment: respects legal, compliance, and data privacy boundaries ## Description Were looking for an experienced Python developer who is strong in web scraping and data extraction to join our data engineering team. Youll build and maintain reliable, scalable pipelines that collect data from websites, APIs, and other online sources. This data feeds analytics, research, and decision-making across our business. This role suits someone who has outgrown writing one-off scripts. You like building resilient, well-tested systems and take pride in data quality. Experience in finance, fintech, or investments is a real advantage, since much of our data supports market, investment, and financial analysis., * Design, develop, and maintain scalable web scraping and crawling solutions in Python * Extract data from static pages, JavaScript-heavy sites, and REST/GraphQL APIs * Build robust pipelines that handle anti-bot measures, rate limits, CAPTCHAs, dynamic content, and site structure changes * Clean, validate, normalize, and transform raw data into analysis-ready datasets * Monitor scraper health, set up alerting, and quickly fix breakages to keep data fresh and accurate * Optimize scrapers for speed, reliability, and cost, including proxy management and concurrency * Store and manage data in relational and non-relational databases and cloud storage * Write clean, tested, well-documented code and take part in code reviews * Work with analysts, data scientists, and business stakeholders to turn data needs into technical solutions * Ensure scraping practices comply with terms of service, robots.txt, and data privacy regulations * Mentor junior developers and help set team standards for scraping and data quality