Software Engineer - Data Platform

eBay Inc.
Amsterdam, Netherlands
about 2 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
4 years minimum
Working hours
Regular working hours
Job source

Tech stack

Java (Programming Language) Application Programming Interfaces (APIs) Artificial Intelligence Big Data Continuous Integration Data Governance Data Infrastructure Distributed Systems Apache Hadoop Python (Programming Language) Data Streaming Apache Spark
+5 more
Containerization Information Technology Apache Flink Data Management Data Pipelines

Job description

  • Build and own core components of eBay’s declarative data platform, enabling automated generation and optimization of data pipelines at scale
  • Design and evolve systems that manage the global data flow graph, including dataset definitions, dependencies, and execution planning
  • Develop platform capabilities for computation graph management, pipeline optimization, and dataset lifecycle orchestration
  • Engineer solutions that systematically reduce data duplication, redundant compute, and operational inefficiencies across thousands of datasets
  • Contribute to long-term platform architecture through design reviews and architecture documents, ensuring scalability, correctness, and resilience
  • Build platform systems that balance performance, cost efficiency, and data correctness, while enforcing governance and compliance requirements
  • Drive operational excellence for platform services, including observability, reliability, and incident response
  • Collaborate with product, infrastructure, and data teams to standardize dataset definitions and improve platform adoption
  • Develop automation and intelligent tooling to improve platform efficiency, including opportunities to leverage AI/agent-driven optimizations
  • Learn and deepen expertise in areas such as declarative systems, distributed computation graphs, data governance, and large-scale platform engineering

Requirements

Do you have experience in Spark?, Do you have a Master’s degree?, Note: This is a Data Platform Engineering role. Familiarity with Spark, Flink, and the Hadoop ecosystem is useful context, but your primary responsibility is building the platform itself - not authoring pipelines., * Strong experience designing and building large-scale distributed systems or platforms (compute, storage, APIs, or orchestration systems)

  • Proven ability to own and deliver complex platform components end-to-end, from design to production
  • Systems thinking mindset with the ability to reason about data flow, dependencies, scaling bottlenecks, and reliability trade-offs
  • Experience building platform abstractions or frameworks, not just consuming them
  • Strong communication skills with the ability to drive alignment across cross-functional engineering teams
  • Curiosity and growth mindset to explore declarative paradigms, optimization systems, and emerging technologies, * Opportunity to solve foundational data platform challenges at massive scale, impacting thousands of datasets and pipelines
  • High-impact work focused on eliminating inefficiencies and reducing operational cost across the data ecosystem
  • Deep technical challenges in declarative systems, graph-based execution, and large-scale optimization
  • Collaborative and inclusive culture with strong emphasis on engineering excellence and knowledge sharing
  • Supportive environment with focus on sustainable operations, on-call balance, and long-term growth, * 4+ years of experience in distributed systems, platform engineering, or data infrastructure
  • Strong proficiency in Java or Python, with experience building production-grade systems
  • Experience with CI/CD, testing, and containerized environments
  • Solid understanding of distributed system design, algorithms, and scalability patterns
  • Familiarity with technologies such as Spark, Flink, or similar systems
  • Experience working with large-scale data ecosystems or data platforms is a plus
  • BS/MS in Computer Science or equivalent practical experience

About the company

At eBay, we’re more than a global ecommerce leader - we’re changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We’re committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work - every day. We’re in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers - and help us connect people and build communities to create economic opportunity for all.

At eBay, we’re more than a global ecommerce leader - we’re changing the way the world shops and sells. Our platform empowers millions of buyers and sellers in more than 190 markets around the world. We’re committed to pushing boundaries and leaving our mark as we reinvent the future of ecommerce for enthusiasts.

Our customers are our compass, authenticity thrives, bold ideas are welcome, and everyone can bring their unique selves to work - every day. We’re in this together, sustaining the future of our customers, our company, and our planet.

Join a team of passionate thinkers, innovators, and dreamers - and help us connect people and build communities to create economic opportunity for all.

We are looking for Software Engineers to join the Golden Data Sets (GDS) Platform team within eBay’s Core Data Platform organization.

The problem we’re solving is significant: eBay currently manages over ten thousands of loosely defined datasets with fragmented pipeline implementations, resulting in widespread data duplication, redundant compute, and runaway operational costs. The GDS Platform addresses this by building a fully declarative data platform - one that accepts dataset specifications, maintains a comprehensive view of the total data flow graph, and automatically produces optimized pipelines to systematically eliminate these inefficiencies at scale.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:34 min

Capabilities of the Apache Spark processing engine

Ayon Roy · LIVE

3:55 min

Infrastructure challenges when combining Kafka with Apache Flink

Bobur Umurzokov · LIVE

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

1:09 min

Evaluating mature stream processing frameworks for production systems

Soroosh Khodami Soroosh Khodami · WWC 2024

2:04 min

Comparing offline data analytics with online stream processing

Artem Volk Artem Volk +1 · WWC 2024

Videos

See all

Related articles

See all