Data Migration & Batch Engineer

Alchemy Software Solutions LLC
Austin, United States of America
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Austin, United States of America

Tech stack

Query Performance
Airflow
Amazon Web Services (AWS)
Databases
Data Definition Language
Data Integrity
ETL
Relational Databases
Python
Microsoft SQL Server
Standard Sql
SQL Stored Procedures
SQL Databases
Informatica Powercenter
Large Language Models
Indexer
PySpark

Job description

· Own schema-drift detection and reconciliation (Red Gate / SQL comparison) - the deterministic ground truth on which every downstream decision rests.

· Build the deterministic rule ladder for table and job dispositions: staging / truncate-and-reload detection, dead-table detection, target-equivalent matching, straight-through repoint, duplicate-job merge.

· Own the ID-collision engine: a full (never sampled) overlap scan across every shared ID space; classification into true collision, phantom and historical overlap; analysis of the resolution strategy; and a non-bypassable validation harness on every script that touches an ID column.

· Own data reconciliation: row counts, column checksums, referential integrity, stored-procedure output parity, business-key parity; staging promotion gates and validated rollback scripts.

· Batch: build the dependency graph, blast-radius (butterfly) scoring, Simple/Medium/Complex classification, and the migration approach per job.

· Own the SLA validation harness - proving every migrated job and every SLA chain runs within its window under combined nightly volume.

· Emit the downstream artifacts: migration scripts, job modifications, rollback scripts, reconciliation specifications.

Requirements

Experience: 8+ years in relational data migration and/or ETL engineering, with cutovers delivered to production. You own the deterministic core of the database and batch workstreams. Two systems have minted identifiers independently for many years across about a million accounts; merging them incorrectly mis-maps an investor''s assets. This seat is where correctness is defended., · SQL Server (2024) - deep: DDL, stored procedures, indexing, referential integrity, query performance.

· Relational data migration at scale - Red Gate, AWS DMS or equivalent; cutover, reconciliation and rollback delivered in production.

· Identity and key-collision resolution - surrogate versus natural keys, reconciliation and historical-mapping tables, preserving referential integrity under a merge. Rare, and highly valued for this role.

· Informatica PowerCenter; ETL dependency analysis; SLA and batch-window modelling.

· Python and strong SQL; data-quality and validation frameworks.

· The discipline to keep correctness-critical steps deterministic while working alongside an LLM reasoning layer.

Working knowledge

· Informatica IDMC and/or Apache Airflow; PySpark.

· Graph queries; handling of regulated / PII data.

About the company

Since our inception in 2013, Alchemy has been dedicated to reshaping organizational performance through innovative IT services. With a vision to empower businesses seeking a transformative edge, we've positioned ourselves at the forefront of digitization and software modernization. Our name reflects our mission: to transmute technology into gold-standard solutions for our esteemed clients. We proudly serve a diverse range of sectors, including IT and ITES, BFSI, Telecom and Media, Automotive, Manufacturing, Energy, Oil and Gas, Real Estate, Retail, Healthcare, and more. With a global footprint spanning the USA, India, Europe, Canada, Singapore, Japan, and parts of Central and West Africa, we harness a unique blend of competencies, frameworks, and cutting-edge technologies. Together, we drive growth and innovation across industries, helping organizations turn their visions into reality.

Apply for this position