Software Engineer - Data Platform

Cogent Inc
San Francisco, CA, United States
1 day ago
Apply on startup.jobs
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$100,000.0 - $300,000.0
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Airflow Batch Processing Big Data Databases Data Integrity Extract Transform Load (ETL) Data Retrieval Software Design Patterns Distributed Systems Graph Database Information Retrieval
+17 more
Search Technologies SQL Databases Workflow Management Systems Large Language Models Database Optimization Backend Data Lakes Kubernetes Information Technology Apache Flink Apache Kafka Search Engines Terraform Stream Processing Data Pipelines Docker Databricks

Job description

We are looking for Founding and Staff-level Engineers to design and implement the foundational pillars of Cogent’s data platform and integration pipeline in order to reliably ingest and normalize enterprise data sources across many corporate and IT systems and then to enable the transformation of that data into a knowledge base that can be leveraged by other GenAI systems., + Your role will involve building data connectors, running data pipelines, and extracting insights and context from our customer’s enterprise environments.

  • You should get excited about building robust and secure backend systems that handle large volumes of data, ETLs, as well as reimagining how enterprise security teams should extract, store and access data.
  • Design highly performant data pipelines and indexing strategies that enable Applied AI use cases like semantic search and retrieval augmented generation
  • Work with Applied AI teams to taxonomize, correlate and deduplicate data across various enterprise sources, and architect indexing, storage and query strategies for data in Cogent’s data lake.
  • Architect the modern data platform for Gen AI
  • The LLM stack and Gen AI landscape is changing everyday. You’ll be front and center of making build vs buy decisions for various tools and services needed by the data platform, and integrating these decisions into our broader architecture and tech roadmap.
  • Implement generalizable design patterns and system components that allow us to scale from 1 integration to thousands of integrations

Requirements

  • 5+ years of professional experience as a hands-on engineer and technical leader leading multiple projects
  • Experience building data pipelines for Information Retrieval Systems (such as knowledge graphs, search engines, or similar) for enterprise use cases
  • Experience in ETL orchestration and workflow management tools (such as Temporal, Airflow) as well as building batch processing and streaming systems with tools like Apache Kafka/Flink
  • Expert in database fundamentals, SQL, data reliability practices and distributed computing
  • Experience working fluently with standard orchestration & deployment technologies like Kubernetes, Terraform, Docker, Databricks, etc in multiple clouds

About the company

Cogent is an Applied AI Lab building the next generation of AI agents for cybersecurity. AI has fundamentally changed how attacks happen, allowing malicious actors to operate at unprecedented speed and scale. Cogent’s “AI Taskforce” assesses petabytes of enterprise data to remediate these issues before critical breaches occur.

To stay at the cutting edge, we blend frontier research with real-world execution. Alongside our core product work, Cogent Research serves as our applied AI lab, providing the research horsepower needed to make truly agentic security workflows a reality.

Since coming out of stealth, Cogent has experienced rapid growth. We partner with Fortune 500 companies to secure some of the most complex production environments in the world.

We’re backed by Greylock and we’ve built a team with the best minds in applied AI. Our team is comprised of people from:

  • Top universities like Stanford, Berkeley, Penn, Duke, Carnegie Mellon, Waterloo
  • Unicorn, high-growth companies like Scale AI, Databricks, Stripe, Tesla, Coinbase
  • World class cybersecurity experts from Wiz, Abnormal AI, Zscaler
  • Preeminent research labs like Deepmind and SAIL

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on startup.jobs
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:07 min

Inspecting default bridge architectures and custom Docker networks

Oliver Seitz Oliver Seitz · World Congress 2025

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

1:52 min

Structuring and scaling the backend engineering team

Stefan Lingler Stefan Lingler +1 · Coffee With Developers

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:34 min

Docker sandbox architecture and microVM environment integration

Manuel de la Peña Manuel de la Peña · World Congress 2026 Europe

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

Videos

See all

Related articles

See all