Data Engineer - Snowflake & Apache Doris

Excelon Solutions
Sunnyvale, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Intermediate
Compensation
$ 123K

Job location

Sunnyvale, United States of America

Tech stack

Query Performance
Airflow
Amazon Web Services (AWS)
Apache HTTP Server
Azure
Cloud Computing
Continuous Integration
Information Engineering
Data Governance
Data Infrastructure
ETL
Data Masking
Data Visualization
Data Warehousing
Database Queries
Dimensional Modeling
Distributed Systems
Document-Oriented Databases
Python
Online Analytical Processing
Online Transaction Processing
Performance Tuning
Role-Based Access Control
Power BI
SQL Databases
Data Streaming
Tableau
Workflow Management Systems
Snowflake
Indexer
GIT
Cloudformation
Containerization
Data Lake
Semi-structured Data
Kubernetes
Infrastructure Automation Frameworks
Star Schema
Real Time Data
Kafka
Terraform
Looker Analytics
Software Version Control
Data Pipelines
Docker

Job description

We are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines and analytics infrastructure. The ideal candidate has hands-on expertise with Snowflake for cloud data warehousing and Apache Doris for real-time OLAP analytics, and is comfortable working across the full data lifecycle - from ingestion to transformation to serving high-performance queries for analytics and BI teams.

Responsibilities

  • Design, build, and maintain robust, scalable ETL/ELT pipelines to ingest structured and semi-structured data from multiple sources into Snowflake and Apache Doris.

  • Architect and optimize data models, schemas, and partitioning strategies in Snowflake and Doris to support high-throughput, low-latency analytical workloads.

  • Tune query performance, indexing, materialized views, and storage layout in Doris for real-time OLAP use cases.

  • Manage Snowflake warehouse sizing, cost optimization, resource monitors, and query performance tuning.

  • Build and maintain data orchestration workflows (e.g., Airflow, dbt, or similar) to automate pipeline scheduling, dependency management, and monitoring.

  • Implement data quality checks, validation frameworks, and observability/alerting for pipeline reliability.

  • Collaborate with data analysts, data scientists, and BI teams to understand reporting requirements and translate them into performant data models.

  • Support migration and integration projects between transactional systems, data lakes, Snowflake, and Doris.

  • Ensure data governance, security, and access control best practices (RBAC, data masking, encryption) across platforms.

  • Document data architecture, pipeline logic, and operational runbooks.

Requirements

7+ years of experience as a Data Engineer or in a similar role.

  • Hands-on production experience with Snowflake - including SQL scripting, warehouse management, Snowpipe, Streams/Tasks, and performance tuning.

  • Hands-on experience with Apache Doris - including cluster architecture (FE/BE nodes), table models (Aggregate, Unique, Duplicate), materialized views, and real-time data loading (Stream Load, Routine Load, Broker Load).

  • Strong SQL skills and experience with data modeling (star/snowflake schemas, dimensional modeling).

  • Proficiency in Python commonly used in data engineering.

  • Experience with workflow orchestration tools (Apache Airflow, dbt, Dagster, or similar).

  • Familiarity with streaming/messaging systems (Kafka, Pulsar, or similar) for real-time data ingestion.

  • Solid understanding of data warehousing concepts, OLAP vs. OLTP design, and distributed systems fundamentals.

  • Experience with version control (Git) and CI/CD practices for data pipelines.

Preferred Qualifications

  • Experience migrating workloads from traditional data warehouses to Snowflake and/or Doris.

  • Familiarity with cloud platforms (AWS, GCP, or Azure) and infrastructure-as-code tools (Terraform, CloudFormation).

  • Experience with containerization/orchestration (Docker, Kubernetes) for deploying Doris clusters.

  • Knowledge of BI/visualization tools (Tableau, Looker, Superset, Power BI).

  • Background in cost optimization and capacity planning for large-scale analytical systems.

  • Contributions to open-source data engineering projects (e.g., Apache Doris community).

Pay: $57.00 - $59.00 per hour

Experience:

  • Data Engineer: 7 years (Required)
  • Snowflake: 4 years (Required)
  • SQL: 4 years (Required)
  • Python: 4 years (Required)
  • Apache Doris : 2 years (Required)

Benefits & conditions

$57 - $59 an hour - Full-time, Contract

Apply for this position