Senior Data Engineer

Omaha Insights
Paris, France
1 day ago
Apply on www.welcometothejungle.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours

Tech stack

Artificial Intelligence Airflow Amazon Web Services Microsoft Azure BigQuery Cloud Computing Continuous Integration Extract Transform Load (ETL) Database Queries Distributed Computing Environment Python (Programming Language) Machine Learning
+13 more
Operational Databases Standard Sql SQL Databases Unstructured Data Parquet Data Logging Snowflake Apache Spark Pyspark Production Code Machine Learning Operations Data Pipelines Databricks

Job description

  • Own the migration from our current Python/Polars, GCS Parquet and Cloud SQL stack to Databricks and PySpark, from architecture through cutover
  • Set the technical direction: storage layout, orchestration, processing patterns, and the standards the team will work to
  • Keep the existing platform running and reliable throughout - clients don’t get an outage while we rebuild

Data pipelines and quality

  • Build and run the pipelines that produce our financial datasets, optimising for performance, reliability and cost
  • Implement financial models and metrics in SQL and PySpark, working with our Head of Product on definitions
  • Own data quality as a system: validation, reconciliation, and alerting that catches breakage before clients do
  • Own monitoring, logging, and incident response for the data platform

AI and document processing

  • Build the pipelines behind our document extraction work - PDFs, filings, news - with our Head of AI
  • Prepare and serve data for AI models, and make their outputs available to downstream systems
  • Support the NL-to-SQL layer with the schema and semantics it depends on, * Databricks specifically. Coming from BigQuery, Snowflake or Redshift is fine - we’ll cover the gap
  • Financial data: statements, market data, time series. If not, you should want to learn why accounting adjustments matter
  • Unstructured data extraction - PDFs, filings, scraping
  • AI/ML pipelines and serving model outputs into production
  • dbt or similar transformation tooling

Requirements

  • 8+ years building production data systems, including at least one platform you owned end to end
  • Experience leading a technical migration or rebuild - you’ve made the architecture calls and lived with them
  • Strong Python: clean, tested, production code
  • Strong SQL: you write complex queries, read execution plans, and think about how data is laid out
  • Hands-on with a distributed processing engine - Spark, PySpark, or equivalent
  • Cloud infrastructure: object storage, distributed compute, containers, CI/CD. GCP, AWS, or Azure
  • Pipeline and orchestration experience - ETL/ELT, Airflow or similar, monitoring, data quality
  • You can explain a technical trade-off to our Head of Product without a whiteboard

Benefits & conditions

  • Competitive salary, performance-based BSPCE
  • Real autonomy and direct impact on the company
  • A collaborative, non-hierarchical environment
  • Access to cutting-edge technology and tools, with lots of budget for AI
  • Full remote in August and during Christmas holidays
  • Regular team events and activities

You’ve owned a data platform, not just contributed to one. You’re happy being the person accountable when it breaks, and you’d rather build the thing than argue about it in a committee.

About the company

Omaha Insights creates proprietary financial data and analytics that give institutional investors an edge. We’re 20 people, expanding across Europe, and scaling our platform to keep up. You’d own that platform end to end, our only data engineer at first, working day to day with our Head of Product and Head of AI.

We want to compute far more data, refresh it in hours instead of days, draw on many more sources, and give our AI systems a foundation they can actually rely on. Getting there means rebuilding the platform over the next 12 months, moving from Python, Polars, GCS Parquet and Cloud SQL to Databricks and PySpark. Your design, no legacy to inherit, decisions made and shipped the same week, then you hire the team around it, with two engineers planned under you in 2027.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.welcometothejungle.com
Prepare application

Good distractions

Loading talks and stories from around this role…