Software Engineer - Big Data

IBM INDIA PRIVATE LTD
San Francisco, CA, United States
1 day ago
Apply on startup.jobs
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
2 years minimum
Compensation
$130,000.0 - $196,500.0
Working hours
Regular working hours
Job source

Tech stack

Java (Programming Language) Amazon Web Services Microsoft Azure Batch Processing Big Data BigTable BigQuery Programming Tools Gradle Apache Hadoop Apache Hive Cloudera
+13 more
Software Engineering SQL Databases Data Processing Google Cloud Cloud Platform System Apache Spark Kubernetes Information Technology Apache Kafka Restful APIs Terraform Apache Beam Docker

Job description

JobPosting MonetaryAmount USD QuantitativeValue 196500 130000 YEAR 2026-06-02T00:00:00Z

LiveRamp is the data collaboration platform of choice for the world’s most innovative companies. A groundbreaking leader in consumer privacy, data ethics, and foundational identity, LiveRamp is setting the new standard for building a connected customer view with unmatched clarity and context while protecting precious brand and consumer trust. LiveRamp offers complete flexibility to collaborate wherever data lives to support the widest range of data collaboration use cases-within organizations, between brands, and across its premier global network of top-quality partners.

Hundreds of global innovators, from iconic consumer brands and tech giants to banks, retailers, and healthcare leaders turn to LiveRamp to build enduring brand and business value by deepening customer engagement and loyalty, activating new partnerships, and maximizing the value of their first-party data while staying on the forefront of rapidly evolving compliance and privacy requirements.

We on the Identity Engineering team maintain the graph processing engine that connects the trillions of different identifiers for consumers and the devices they use online. Help us process this trillion edge graph as quickly and efficiently as possible.

We handle product requests and enhancements, focusing on scalability, modernization, performance, cost, security, legal compliance, reliability, and quality. Projects include architectural refreshes like POCs for batch data processing, developing new REST APIs, and migrating applications to Spark and SingleStore. These initiatives aim to improve processing speed, scalability, maintainability, and cost-effectiveness.

You will:

  • Lead complex problems, by breaking into their essential components, design and implement elegant solutions focussing on value delivered.
  • Deliver on the problems problems by collaborating with the team and cross functional teams
  • Modernize our applications to be modular, scalable by leveraging the latest stack
  • Collaborate closely with product managers, and other engineers to deliver high-quality software
  • Design with privacy and security-first principles and err on the side of shipping fast and iterating.

Requirements

  • 6+ years of professional software development experience, primarily focused on big data technologies.
  • 3+ years of experience writing and optimizing large-scale batch processing using Spark/Hadoop or similar
  • 3+ years of Java , SQL programming skills
  • 2+ years of experience using cloud based platforms such as GCP, AWS, Azure or similar technologies.
  • Experience with technologies like Docker, Terraform and/or Kubernetes
  • Startup personality and enjoy working as part of a team: smart, ethical, friendly, hard-working, and productive.
  • Bachelor’s degree in a quantitative field such as Statistics, Mathematics, Computer Science, Engineering, Economics, or a related discipline.

Nice to have:

  • Experience with GCP products such as BigTable, BigQuery, Dataproc, PubSub or other data tools like Kafka, Hive, or Apache Beam
  • Experience with development tools such as Terraform, Kubernetes, Helm, or Gradle
  • Experience with designing and implementing interfaces and infrastructure for large scale systems and RESTful APIs

Benefits & conditions

The approximate annual base compensation range is $130,000 to $196,500. The actual offer, reflecting the total compensation package and benefits, will be determined by a number of factors including the applicant’s experience, knowledge, skills, and abilities, geography, as well as internal equity among our team., * People: Work with talented, collaborative, and friendly people who love what they do.

  • Fun: We host in-person and virtual events such as game nights, happy hours, camping trips, and sports leagues.
  • Work/Life Harmony: Flexible paid time off, paid holidays, options for working from home, and paid parental leave.
  • Comprehensive Benefits Package: LiveRamp offers a comprehensive benefits package designed to help you be your best self in your personal and professional lives. Our benefits package offers medical, dental, vision, life and disability, an employee assistance program, voluntary benefits as well as perks programs for your healthy lifestyle, career growth and more.
  • Savings: Our 401K matching plan-1:1 match up to 6% of salary-helps you plan ahead. Also Employee Stock Purchase Plan - 15% discount off purchase price of LiveRamp stock (U.S. LiveRampers)

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on startup.jobs
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:07 min

Inspecting default bridge architectures and custom Docker networks

Oliver Seitz Oliver Seitz · World Congress 2025

5:51 min

Understanding build structure and core Gradle plugins

Amanda Martin · LIVE

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:34 min

Docker sandbox architecture and microVM environment integration

Manuel de la Peña Manuel de la Peña · World Congress 2026 Europe

2:14 min

Solving complex platform architecture challenges at an enterprise scale

Maria Apazoglou · Coffee With Developers

Videos

See all

Related articles

See all