Data Engineer

Diligent Robotics
Glastonbury, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Glastonbury, United States of America

Tech stack

API
Business Logic
Google BigQuery
Software as a Service
Information Engineering
Data Warehousing
Dimensional Modeling
Identity and Access Management
Operational Data Store
Operational Databases
SQL Databases
Data Ingestion
Snowflake
Data Management

Job description

We're hiring our Founding Senior Data Engineer - the first dedicated data hire on a team that already has active analytics demand, working dashboards, and a clear plan for what comes next. You'll own the design and build of our BigQuery data warehouse, set the standards the rest of the data function will follow, and partner closely with our existing analyst team to build the analytics platform that will power the next stage of the business., * Stand up the warehouse. Design and implement our BigQuery environment from the ground up - project structure, IAM, dataset organization, naming conventions, cost controls - so it scales cleanly as we add sources and users.

  • Build the transformation layer. Set up dbt as the home for our transformation logic. Migrate the business logic from our existing analytics environment into version-controlled, tested dbt models.
  • Model master data. Introduce dimensional models for our core business entities so data from different sources can be joined consistently.
  • Own ingestion. Configure native Firestore-to-BigQuery integration, then stand up Airbyte as our managed connector layer for SaaS sources and log exports as we broaden the warehouse.
  • Partner across the org. Work with source-system owners, the analyst team, and engineering teams that will eventually consume curated data from the warehouse. Help define the boundary between what the warehouse is and is not for.

Requirements

  • 5+ years of hands-on data engineering experience, including significant work in a modern data stack (cloud warehouse + dbt + version-controlled transformations).
  • Deep, hands-on experience with BigQuery (or another columnar cloud warehouse - Snowflake, Redshift - with willingness to ramp on BigQuery quickly).
  • Deep, hands-on experience with dbt, including project structure, testing, and modeling patterns. You can debate the tradeoffs of staging vs. intermediate vs. marts layering and have opinions about when to use incremental models.
  • Strong dimensional modeling fundamentals (Kimball or equivalent). You understand conformed dimensions, slowly changing dimensions, and grain.
  • Strong SQL. Comfortable reviewing and refactoring complex SQL written by analysts.
  • Experience designing ingestion pipelines from multiple source types - operational databases, SaaS APIs, log-based sources.
  • Excellent technical communication. You can explain tradeoffs and modeling decisions to analysts, engineers, and executives.
  • Comfortable being the first dedicated data engineer in an organization - setting standards, making decisions without a peer to defer to, partnering with non-data folks., * Experience with Firestore and the native Firestore-to-BigQuery integration, particularly at scale (multi-project setups).
  • Experience with Airbyte (self-hosted or cloud) or comparable managed ingestion tools (Fivetran, Stitch).
  • Experience working with Elastic as a data source - extracting metrics or operational data out of Elastic indexes into downstream systems.
  • Experience designing a metrics catalog or semantic layer (dbt Semantic Layer, MetricFlow, LookML, or similar).

Apply for this position