Lead Data Engineer

• Team Leadership: Lead engineers, review code, establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging). Leadership experience is mandatory.

• Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions.

• Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage.

• Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost management (e.g., slot reservations, partitioning).

• Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases.

• Data Quality & Governance: Implement data validation, lineage, and metadata management (e.g., Data Catalog, dbt).

• Technical leadership experience is mandatory. We are looking for a lead, not a senior Data engineer. Candidates without 5+ years of experience as lead will not be considered.

• Technical: Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam).

• GCP Expertise: Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform.

• Cloud SQL, Cloud Storage and GCP Suite, Cloud Run (Heavy) and custom templates

• Kafka (based upon data source) Data Plex is use with Every workflow.

• Leadership: Experience leading projects, mentoring teams, and driving technical roadmaps.

• DevOps: Familiarity with CI/CD, Git, and infrastructure-as-code.

• Leads help shape backlog and user stories (team then owns/executes)., * GCP Certifications (e.g., Professional Data Engineer).

• Experience with real-time processing, Vertex AI, or multi-cloud environments.

• DBT Labs & understanding of clinical/CMS billing data (context for latter: resources will be working with full medallion layers in cloud.).

The pay range that the employer in good faith reasonably expects to pay for this position is $42.07/hour - $65.73/hour. Our benefits include medical, dental, vision and retirement benefits. Applications will be accepted on an ongoing basis.