Data Engineer
McClure Engineering Co.
New York, United States
8 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on startup.jobs
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours
Job source
Tech stack
Query Performance
Artificial Intelligence
Airflow
Apache HTTP Server
Data Infrastructure
Data Security
Data Sharing
Software Debugging
Disaster Recovery
Identity and Access Management
Python (Programming Language)
Operational Databases
+12 more
Performance Tuning
Software Maintenance
Blockchain
Data Streaming
Workflow Management Systems
Delivery Pipeline
Snowflake
Apache Spark
Change Data Capture
Apache Kafka
Database Replication
Data Pipelines
Job description
Build and run the internal data platform that helps Alchemy understand product usage, revenue, and the cost of serving customers. The existing Data Engineer intake centers on ingestion, Snowflake operations, workflow orchestration, data quality, and collaboration with analytics partners.
You will own ingestion, schema changes, and checks that show whether shared datasets are current and correct. You will work with analytics engineers and data scientists to define those datasets for reporting, analysis, and AI-assisted tools.
What You Will Do
- Build batch and streaming pipelines that connect operational databases, product events, billing systems, and external sources to the internal data platform.
- Make ingestion recoverable through change-data capture, schema-change handling, idempotent processing, and safe backfills.
- Operate and improve Snowflake, including access controls, workload management, query performance, and compute and storage costs.
- Improve orchestration and deployment workflows so engineers can test, ship, and debug pipelines safely.
- Build data-quality checks that detect stale, missing, duplicated, or inconsistent records before they reach downstream consumers.
- Work with analytics partners on shared data models, source definitions, and documentation. Make dataset ownership and upstream dependencies clear.
- Reduce recurring manual work and build reusable tools for self-service data access.
- Participate in production support and incident response for pipelines and the data platform. Restore data flow, reconcile affected datasets, and fix the causes of recurring failures., * You own a production pipeline and have shipped an improvement to its freshness, reliability, or operating cost.
- Downstream users can see whether its data is current, recovery procedures have been tested, and a recurring manual request has become a reusable dataset or tool.
Requirements
- Experience independently building and operating production data pipelines and the infrastructure around them.
- Strong SQL and Python skills, including testing, debugging, and maintainable software design.
- Experience with Snowflake or a comparable warehouse, including performance tuning and access management.
- Experience with workflow orchestration and transformation tools such as Airflow and dbt, or equivalent systems.
- Understanding of data modeling, incremental processing, schema evolution, and data-quality testing.
- The ability to trace a data discrepancy from a downstream report back to its source, explain the cause, and fix it.
- Good judgment about stakeholder needs: turn recurring requests into reusable datasets or tooling, and make trade-offs clear.
Blockchain experience is not required.
Helpful Experience
- Kafka, database replication, or change-data-capture systems.
- Apache Iceberg, Spark, Trino, or other lakehouse and query technologies.
- Infrastructure as code, data observability, lineage, and disaster recovery.
- Usage metering, billing and revenue data, or semantic layers for self-service analytics.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on startup.jobs
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦