Senior Data Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+13 more
Job description
- Own the migration from our current Python/Polars, GCS Parquet and Cloud SQL stack to Databricks and PySpark, from architecture through cutover
- Set the technical direction: storage layout, orchestration, processing patterns, and the standards the team will work to
- Keep the existing platform running and reliable throughout - clients don’t get an outage while we rebuild
Data pipelines and quality
- Build and run the pipelines that produce our financial datasets, optimising for performance, reliability and cost
- Implement financial models and metrics in SQL and PySpark, working with our Head of Product on definitions
- Own data quality as a system: validation, reconciliation, and alerting that catches breakage before clients do
- Own monitoring, logging, and incident response for the data platform
AI and document processing
- Build the pipelines behind our document extraction work - PDFs, filings, news - with our Head of AI
- Prepare and serve data for AI models, and make their outputs available to downstream systems
- Support the NL-to-SQL layer with the schema and semantics it depends on, * Databricks specifically. Coming from BigQuery, Snowflake or Redshift is fine - we’ll cover the gap
- Financial data: statements, market data, time series. If not, you should want to learn why accounting adjustments matter
- Unstructured data extraction - PDFs, filings, scraping
- AI/ML pipelines and serving model outputs into production
- dbt or similar transformation tooling
Requirements
- 8+ years building production data systems, including at least one platform you owned end to end
- Experience leading a technical migration or rebuild - you’ve made the architecture calls and lived with them
- Strong Python: clean, tested, production code
- Strong SQL: you write complex queries, read execution plans, and think about how data is laid out
- Hands-on with a distributed processing engine - Spark, PySpark, or equivalent
- Cloud infrastructure: object storage, distributed compute, containers, CI/CD. GCP, AWS, or Azure
- Pipeline and orchestration experience - ETL/ELT, Airflow or similar, monitoring, data quality
- You can explain a technical trade-off to our Head of Product without a whiteboard
Benefits & conditions
- Competitive salary, performance-based BSPCE
- Real autonomy and direct impact on the company
- A collaborative, non-hierarchical environment
- Access to cutting-edge technology and tools, with lots of budget for AI
- Full remote in August and during Christmas holidays
- Regular team events and activities
You’ve owned a data platform, not just contributed to one. You’re happy being the person accountable when it breaks, and you’d rather build the thing than argue about it in a committee.
About the company
Omaha Insights creates proprietary financial data and analytics that give institutional investors an edge. We’re 20 people, expanding across Europe, and scaling our platform to keep up. You’d own that platform end to end, our only data engineer at first, working day to day with our Head of Product and Head of AI.
We want to compute far more data, refresh it in hours instead of days, draw on many more sources, and give our AI systems a foundation they can actually rely on. Getting there means rebuilding the platform over the next 12 months, moving from Python, Polars, GCS Parquet and Cloud SQL to Databricks and PySpark. Your design, no legacy to inherit, decisions made and shipped the same week, then you hire the team around it, with two engineers planned under you in 2027.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…