Data Engineer

Randstad

Charlotte, NC, United States

about 1 month ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Compensation

$124,800.0 - $137,280.0

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Adobe InDesign Artificial Intelligence Amazon Web Services Amazon S3 Data Analysis Unit Testing CA Workload Automation Ae Batch Processing Big Data Unix Computer Programming Databases

+32 more

Information Engineering Extract Transform Load (ETL) Data Mining Data Profiling Data Warehousing Relational Databases IBM InfoSphere DataStage DevOps Github Python (Programming Language) PostgreSQL Microsoft SQL Server Oracle (Applications) Oracle SQL Developer Pentaho Data Integration Performance Tuning Scrum Methodology SQL Stored Procedures PL-SQL SQL Databases Data Processing Informatica Powercenter Sybase Pyspark Data Analytics Apache Kafka Bitbucket Restful APIs Data Pipelines Sql Tuning Jenkins Control M

Job description

The primary role of the Data Engineer is to function as a critical member of a data team by designing data integration solutions that deliver business value in line with the company’s objectives. They are responsible for the design and development of data/batch processing, data manipulation, data mining, and data extraction/transformation/loading into large data domains using Python/Pyspark and AWS tools.

Requirements

Minimum 8 years of hands-on data engineering experience in development using Python and Pyspark

Experience in AWS cloud services like Glue, Lambda, MSK (Kafka), S3, Step functions, RDS, EKS
Experience in Databases like Postgres, SQL Server, Oracle, Sybase
Experience with SQL, performance tuning, queries, stored procedures, views, functions, and triggers
Ability to learn and use AI tools effectively
Understanding of Data pipelines, ETL processes, CI/CD DevOps process and tools like GitHub, Jenkins
Knowledge of Data modeling, Warehousing concepts and Agile/SCRUM methodology
Strong problem-solving and analytical skills

qualifications:

-Developer experience specifically focusing on Data Engineering.

-Hands-on experience in Development using Python and Pyspark as an ETL tool.

-Experience in AWS services like Glue, Lambda, MSK (Kafka), S3, Step functions, RDS, EKS etc.

-Experience in Databases like Postgres, SQL Server, Oracle, Sybase etc.

-Experience with SQL database programming, SQL performance tuning, relational model analysis, queries, stored procedures, views, functions, and triggers.

-Strong technical experience in Design (Mapping specifications, HLD, LLD), Development (Coding, Unit testing).

-Knowledge in developing UNIX scripts, Oracle SQL/PL-SQL.

-Experience with data models, data mining, data analysis and data profiling.

-Experience in working with REST API’s.

-Experience in workload automation tools like Control-M, Autosys etc.

-Good knowledge in CI/CD DevOps process and tools like Bitbucket, GitHub, Jenkins.

-Strong experience with Agile/SCRUM methodology.

-Experience with other ETL tools (DataStage, Informatica, Pentaho, etc.).

-Knowledge in MDM, Data warehouse and Data Analytics.