Data Engineer

Nous Research, Inc.

United States

24 days ago

Apply on www.indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.indeed.com

Tech stack

Artificial Intelligence Airflow Databases Information Engineering Data Infrastructure Extract Transform Load (ETL) Github Identity and Access Management Python (Programming Language) Operational Databases SQL Databases Working Model 2D

+2 more

Large Language Models Data Pipelines

Job description

Build, operate, and continuously improve the data pipelines that feed analytics and model training.
Deliver reliable internal analytics so teams across Nous can make decisions on their own data.
Wrangle long-horizon agentic data into structured, well-documented datasets ready for analysis and training.
Choose ETL/ELT approaches based on destination capabilities, reproducibility, latency, and cost, and explain the tradeoffs behind each choice.
Own pipeline operations in production, including failures, re-runs, and backfills, and debug issues across jobs, storage, permissions, and downstream consumers.
Design and evolve schemas, storage formats, and data models as needs change.
Build memory-efficient processing that scales, using streaming and partitioning, and recognize when a workload has outgrown a single machine.
Communicate clearly about data location, lineage, quality, limitations, and fitness for use to engineers and non-engineers alike.

Requirements

4+ years of data engineering experience building and operating production data pipelines.
Strong fundamentals in SQL, data modeling, and practical schema design and evolution.
Deep understanding of storage formats and database types, and when each one fits.
A working model of the data stack: ingestion, storage, transformation, orchestration, cataloging, and serving, and where responsibilities belong in each layer.
Strong Python, plus solid shell skills for batch and storage operations.
Hands-on cloud experience with object storage, batch compute, and identity and access management.
Production experience with at least one workflow orchestration system such as Airflow, Dagster, Prefect, or an in-house equivalent.
An instinct for data quality: you ask “is the data good?” and not just “did the pipeline run?”
Extensive use of AI-assisted development tools, with disciplined review of what those tools produce., * Experience working with agentic or LLM-generated data, including structured, multi-step, tool-conditioned interactions.
Experience building data pipelines for model training.
Experience processing datasets at the scale of millions of interactions or more.
Background at data infrastructure, observability, or AI data companies.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Prepare application

Good distractions

Loading talks and stories from around this role…