Senior Data Engineer

We're looking for a Senior Data Engineer to take ownership of the core data platform, designing and building systems that aggregate, process, and serve data from thousands of third-party websites.

This is a high-impact role with significant responsibility for both architecture and implementation.

What You'll Work On

• Designing a system to extract data from thousands of heterogeneous websites
• Building pipelines for continuous, reliable data updates
• Delivering structured data via a low-latency API layer
• Implementing efficient caching strategies (minimal long-term storage)

Key Challenges

• Handling site variability at scale
• Working around anti-bot protections and rate limiting
• Designing for resilience, monitoring, and failure recovery
• Keeping infrastructure lean and cost-efficient, * Python (Playwright, Scrapy, Selenium)
• APIs (FastAPI, Django)
• Queues / orchestration (Celery, Kafka, RabbitMQ)
• Cloud infrastructure (AWS, GCP, etc.)

What We Offer

• A foundational engineering role in a growing platform
• High degree of ownership and autonomy
• Close collaboration with decision-makers
• Compensation is competitive for a senior hire and includes equity

Not the Right Role If

• You prefer tightly scoped tickets over open-ended problems
• You're looking for a purely maintenance role
• You're early in your career

Apply With

• Examples of relevant systems or projects
• A short outline of how you would approach this problem
• Your availability

You're a senior engineer who:

• Has built or contributed to large-scale scraping or data ingestion systems
• Is comfortable owning architecture as well as implementation
• Thinks in terms of systems, trade-offs, and long-term maintainability
• Can communicate clearly with non-technical stakeholders