Lead DATA ENGINEER - Cloud Migration

TechDigital Corporation
Jersey City, NJ, United States
1 day ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours

Tech stack

Artificial Intelligence Data Analysis Cloud Computing Cloud Computing Security Encodings Data Architecture Information Engineering Data Migration Apache Hive Role-Based Access Control Software Engineering Management of Software Versions
+11 more
Enterprise Data Management Cloud Platform System Apache Spark Agentic-AI Data Lakes Pyspark Data Management Machine Learning Operations Cloud Migration Code Restructuring Databricks

Job description

This role is part of a multiyear enterprise initiative to modernize data platforms by migrating from legacy and onprem environments to cloudnative, governed, and scalable architectures . The role focuses on migrating enterprise data workloads to Databricks on strategic cloud platforms , enabling standardized data engineering, analytics, centralized reporting, reconciliation utilities, and AI/ML use caseswhile adhering to controls, security, resilience, and regulatory compliance ., * Lead and execute migration of legacy data platforms (onprem / nonstandard tools) to Databricks on cloud under the Olympus program

  • Perform application, data, and pipeline refactoring to cloudnative Databricks patterns
  • Drive migration planning including dependency analysis, sequencing, and cutover strategy
  • Support coexistence models and transition from dualrun to cloudonly execution

Databricks Lakehouse Engineering

  • Design and implement Databricks Lakehouse architecture (Bronze / Silver / Gold)
  • Build scalable batch and streaming pipelines using PySpark, Spark SQL
  • Leverage Delta Lake for reliability, versioning, and performance
  • Optimize compute usage and cost in line with enterprise cloud efficiency goals

Enterprise Data Controls & Governance

  • Embed data quality, reconciliation, and completeness controls as part of migration
  • Ensure migrated workloads meet EDO governance, MCA, and audit requirements
  • Maintain lineage, traceability, and explainability across migrated assets
  • Support riskcritical use cases (Finance, Ops, Recon, Reporting)

Cloud Security & Resilience

  • Implement cloudaligned RBAC, identity controls, and secure access patterns
  • Enforce data encryption, masking, and classification standards
  • Ensure workloads meet operational resilience and recovery expectations
  • Partner with cloud platform and security teams for certification and signoff

Reporting, Analytics & AI Enablement

  • Enable downstream BI, regulatory reporting, and MI workloads on Databricks
  • Support centralized reporting programs (e.g., ARA, GRUrelated use cases)
  • Prepare data foundations for AI / ML and Agentic workflows postmigration

Requirements

Software Engineering, Cloud Computing, Reconciliation, Regulatory Compliance, Software Engineering, Cloud Computing, Artificial Intelligence (AI), Data Migration, Data Analysis, Refactoring, * 812+ years in data engineering / platform modernization

  • Strong handson experience with Databricks in largescale enterprises
  • Proven experience delivering cloud migration programs (onprem cloud)
  • Deep expertise in Apache Spark, PySpark, Spark SQL
  • Experience embedding controls, reconciliation, and data quality in migrations
  • Experience in regulated environments (banking / financial services preferred), * Experience with Client Olympus or equivalent enterprise cloud programs
  • Knowledge of legacy data platforms and modernization patterns
  • Familiarity with Finance, Ops, Recon, or BalanceSheet data domains
  • Exposure to MLflow, AI pipelines, or GenAI enablement on cloud
  • Strong understanding of runthebank vs changethebank execution

Behavioral & Delivery Expectations

  • Strong ownership and execution mindset
  • Comfortable operating in large, multivendor transformation programs
  • Ability to engage with Technology, Operations, Risk, and Audit stakeholders
  • Disciplined approach to migration risk, controls, and documentation

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…