Ab Initio Developers

J&C Associates

Northampton, UK

19 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

www1.jobdiva.co.uk

Tech stack

Java (Programming Language) Agile Methodology Automation of Tests Big Data Cloudera Impala Continuous Integration Data Governance Extract Transform Load (ETL) Data Security Data Systems Relational Databases DevOps

+25 more

Apache Hadoop Hadoop Distributed File System Apache HBase Apache Hive Python (Programming Language) Logical Data Models Metadata Repositories Oracle Databases Mockito Standard Sql Cloudera Scala (Programming Language) Service Layer Software Engineering Teradata SQL Apache Zookeeper Enterprise Data Management Data Storage Technologies Apache Yarn Apache Spark Ab Initio Data Lakes Deployment Automation Data Management Physical Data Models

Job description

Ab Initio ETL Designer, you will be able to develop complicated Ab Initio ETL processes to turn business requirements into Technology data solutions.
You will provide technically sound solutions for the ingestion, storage and presentation of enterprise data including ETL design, data storage strategies, data access and security in line with Business requirements/processes and create Analytical Ready Data.
You will drive the design of scalable solutions while considering recoverable and resiliency requirements.
You will employ agile techniques such as task estimation, test automation, deployment automation and continuous integration to enhance overall execution speed and product quality.
You will convert source to target mappings into ETL code.
You will collaborate with the data modellers and contribute to the physical data model design.
You will ensure to work in line with Global Data Management Standards and follow all Governance Controls as required for your work.
SOX/DQ - Performing gap analysis between existing SOX/DQ controls and the data governance/data reconciliation.
Lineage- Work with SOX Tech delivery team and ensure the lineage captured and Reconciliation between hops are documented.
Design and implement physical data and metadata repositories in Hive/Impala/HBase (as appropriate) corresponding to logical data models
Design and implement data service layer in Hadoop
Replicate data from current RDBMS based systems to the RFT Hadoop Data Lakes
Utilise Dev-Ops framework during the development activities

Requirements

Passion for programming and software development
Extensive knowledge of Ab Initio ETL development in Big Data world specially in Hadoop eco system
Knowledge of Hadoop Ecosystem- Cloudera distribution is preferred. Knowledge of key Hadoop components mainly HDFS, Zookeeper, YARN, HBASE, Hive, Impala, Spark SQL
Knowledge of software development using Python, Scala, Spark and Java is critical to this role.
Usage of DevOps tools and knowledge of it is key to success in this role.
Working knowledge on Teradata and Oracle databases.
Experience in creating Narrative documents for Governance to prove existing Controls work.
Experience in data modelling with good SQL skills in at least 1 enterprise grade RDBMS
Experience in Test/Behaviour Driven Development (including test automation and mocking tools)
Strong project and stakeholder management skillset.
Has an eye for data quality issues and providing solution to resolve and remediate