ETL/Data Engineer

BOGART AND ASSOCIATES INC

Huntsville, AL, United States

2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

clearancejobs.com

Tech stack

Java (Programming Language) Amazon Web Services Amazon S3 Databases Data Validation Data Integration Extract Transform Load (ETL) Data Storage Technologies Apache Spark Information Technology AWS Glue Data Lakehouse

+3 more

Data Pipelines Amazon Redshift Databricks

Requirements

8+ years of IT experience

Strong proficiency in programming languages such as Scala, or Java.
Experience with data validation, cleansing, and enrichment techniques to improve the accuracy and completeness of data.
Experience monitor and troubleshoot ETL processes to identify and resolve issues in a timely manner.
Experience in designing and developing ETL workflows using tools like Apache Spark or AWS Glue.
In-depth knowledge of ETL best practices, data integration techniques, and data quality management.
Familiarity with different data storage technologies and databases, such as Amazon S3 or Amazon Redshift
Build and optimize data pipelines to extract data from various sources, transform it into the required format, and load using Databricks and AWS services.
Bachelor’s degree Preferred:
Understanding of concepts of Data Lakehouse architecture is a plus
Experience monitoring and troubleshooting ETL processes to identify and resolve issues in a timely manner.