DATA ENGINEERS

Ipolarity LLC
Piscataway, NJ, United States
3 days ago
Apply on www.juju.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$166,754
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Oracle (Applications) Data Processing Snowflake Ab Initio Data Lakes Pyspark Data Lineage Terraform Databricks

Job description

Design and implement automation frameworks underpinning the migration. Build reusable transformation component libraries covering parameterized ingestion, SCD processing, data quality assertion, and reconciliation reporting. Construct end-to-end column-level data lineage and data flow diagrams. Identify and eliminate data processing redundancies and perform pipeline inventory analysis. Implement Terraform IaC managing all Databricks and AWS resources. Execute Ab Initio legacy pipelines in parallel with new PySpark/Databricks equivalents. Track lineage from Oracle/Snowflake source systems through Ab Initio transformation layers and PySpark pipelines to Delta Lake Gold consumption tables.

Requirements

Bachelor’s degree in Science, Technology, or Engineering (any) with 5 years of experience in the job offered or related occupation is required. Work location: Piscataway, NJ and various unanticipated locations throughout the U.S.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.juju.com
Prepare application

Good distractions

Loading talks and stories from around this role…