AB initio Developer - Onsite role
Saransh Inc
Jersey City, NJ, United States
5 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.thejobnetwork.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
Tech stack
Amazon Web Services
Amazon S3
Cloud Computing
Directed Acyclic Graph (Directed Graphs)
Information Engineering
Data Integration
Extract Transform Load (ETL)
Distributed Systems
Python (Programming Language)
Data Logging
Data Processing
Apache Spark
+3 more
Ab Initio
Pyspark
Data Pipelines
Job description
- Creating efficient and scalable data processing pipelines and applications using Apache spark.
- Implementing, scheduling and monitoring complex data workflows, ensuring timely and accurate data processing.
- Implement and manage agents for monitoring, logging, and automation within aws environments
- Handling migration from Pyspark to AWS
Requirements
- Experience in building data integration solutions using the AB initio ETL .
- Creating efficient and scalable data processing pipelines and applications using Apache spark
- Proficient in understanding distributed computing principles.
- Creating ,optimizing, and maintaining directed acyclic graphs(DAGs) in Python to define and orchestrate data pipelines and automated tasks.
- Implementing , scheduling, and monitoring complex data workflows, ensuring timely and accurate data processing.
- Identifying, diagnosing, and resolving issues within airflow work flows, and optimizing DAGs for performance, scalability, and resource efficiency.
- Good understanding of cloud technology. Must have strong technical experience in Desing Mapping Specifications HLD LLD.
- Must have the ability to relate to both business and technical members of the team and possess excellent communication skills.
- Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, EMR and integrate witho our internal Archival Service Platform for efficeient data purging and LifeCycle management.
- Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.thejobnetwork.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦