Data Engineer

Algo Soft Solutions LLC
New York, NY, United States
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours
Job source

Tech stack

Sql Data Warehouse Application Programming Interfaces (APIs) Amazon Web Services Business Analytics Applications Automation of Tests Cloud Computing Databases Data Integration Extract Transform Load (ETL) Data Migration Data Systems Data Warehousing
+26 more
DevOps Event-Driven Programming Github Python (Programming Language) Release Management Salesforce.Com SQL Databases Systems Integration Real Time Systems Delivery Pipeline Snowflake Test Scripts Change Data Capture Event Driven Architecture Build Management Information Technology AWS Fargate AWS Data Analytics Data Management Tools for Reporting Remote API Restful APIs Terraform Oracle Integration Data Pipelines Serverless Computing

Job description

The Data Management and Engineering group builds modern data and analytics platforms for the business. We are hiring a Data Engineer to design and build integration solutions supporting a broader modernization push. The team is newly formed and growing fast, so this is a ground floor opportunity with an organization that has expanded steadily over the last three years.

What You’‘ll Build

· Translate business requirements into practical data solutions

  • Serverless pipelines on AWS to move and transform data at scale
  • Event driven architecture for decoupled, reliable data movement
  • Real time system to database integrations using standardized frameworks
  • Self service analytics tools giving business users access across multiple source systems
  • CI/CD pipelines using Terraform and GitHub Actions across environments

Core Responsibilities Pipeline work: build production grade pipelines on AWS serverless tools with solid error handling, retry logic, and observability. Implement data warehousing for historical tracking and compliance. Build incremental processing with change data capture and idempotent design. Create multi stage orchestration with parallel execution. Write and run test scripts before production release.

Integration work: connect enterprise systems (ERP, CRM, ecommerce, third party APIs) to each other and to the cloud data warehouse. Build event driven flows using AWS messaging and streaming tools. Design RESTful APIs with proper authentication for real time access. Build integrations spanning retail, supply chain, and finance systems.

Infrastructure and DevOps: manage cloud resources as code using Terraform. Build multi environment deployment pipelines with automated testing. Set up monitoring and alerting for pipeline health, data quality, and SLA tracking. Optimize cloud costs through serverless design and right sizing. Define security guardrails for data integrations.

Requirements

Requirement Notes (Candidate Job description below) : We need a mid-senior Data Engineer with experience building AWS Data Pipelines with (5 to 7+ years) designing and supporting data integrations and pipeline orchestration on AWS (Lambda, Step Functions, Fargate, Glue, etc). Snowflake experience required, 5 plus years. 3 plus years with SQL and Python. Retail industry exposure a plus. Experience with an integration platform like Oracle Integration Cloud a plus. Reporting tool experience such as Sigma a strong plus. Salesforce ecosystem exposure preferred (SFCC, SFSC, SFMC). Strong written and verbal communication skills. ** CANDIDATES CAN BE MIDLEVEL WITH 5+ YEARS AND HAVE GOOD AWS DATA PIPELINE EXPERIENCE - SNOWFLAKE IS A BIG PLUS BUT PLEASE STILL SEND CANDIDATES IF THEY DO NOT HAVE SNOWFLAKE - CERTIFICATIONS PREFERRED.

**CANDIDATES MUST HAVE GOOD TENURE AND/OR LONG PROJECTS. CANDIDATES WITH RECENT LOCAL PROJECTS HIGHLY PREFERRED. - NO RELOCATION, 5 to 7 years designing and supporting data integrations and pipeline orchestration on AWS (Lambda, Step Functions, Fargate, Glue, etc). Snowflake experience required, 5 plus years. 3 plus years with SQL and Python. Retail industry exposure a plus. Experience with an integration platform like Oracle Integration Cloud a plus. Reporting tool experience such as Sigma a strong plus. Salesforce ecosystem exposure preferred (SFCC, SFSC, SFMC). Strong written and verbal communication skills.

Education Master’’s in computer science, or bachelor’’s with equivalent experience. Snowflake or Salesforce certifications a plus

AWS shop

Lambda

Data team. They do date integration. Lots of APIs. Moving data to other locations.

Database is SnowFlake

DMS- data migration services

Retail knowledge is helpful but not necessary

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:17 min

Mapping the maturity roadmap for scaled devops adoption

Dominik Krichbaum Dominik Krichbaum · WWC Europe 2026

6:36 min

Funding open source through GitHub Accelerator and Sponsors

Stormy Peters · WWC 2023

1:34 min

Essential commands for running and testing Terraform configurations

Hennie Francis · LIVE

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

3:18 min

Scaling global network engineering through DevOps culture

Stuart Clark · LIVE

Videos

See all

Related articles

See all