Hadoop Cloudera Engineer

First Call Trading Corporation

Irving, TX, United States

3 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

disabledperson.com

Tech stack

Query Performance Amazon Web Services Microsoft Azure Big Data Cloud Computing Cloudera Impala Computer Programming Continuous Integration Data Governance Extract Transform Load (ETL) Data Security Data Warehousing

+31 more

DevOps Apache Hadoop Hadoop Distributed File System MapReduce Apache HBase Apache Hive JSON Python (Programming Language) Performance Tuning Cloudera Simple Data Format SQL Databases Sqoop Unstructured Data Parquet Data Processing Data Storage Technologies Real Time Systems Data Ingestion Apache Yarn Apache Spark Git Pyspark Apache Flume Kubernetes Avro Apache Kafka Spark Streaming Data Management Software Version Control Docker

Job description

Develop and maintain big data solutions using Cloudera Hadoop platform

Design and implement scalable ETL pipelines for large datasets

Work on data ingestion, processing, and transformation using Spark and Hadoop tools *Optimize data workflows, query performance, and storage strategies

Manage and maintain Hadoop clusters (Cloudera distribution)

Work with structured and unstructured data across multiple sources

Collaborate with Data Engineers, Analysts, and Business teams

Implement data security, governance, and access controls

Ensure system performance, reliability, and scalability Collaborate with cross-functional teams (Data Engineers, DevOps, Business teams)

Requirements

Hadoop & Cloudera , Ecosystem (CDP/CDH, HDFS, YARN, MapReduce)

Spark & Data Processing(Spark Core, SQL, PySpark)

Version Control (Git)

ETL Pipeline Development &Optimization

Cloud & DevOps (CI/CD, Docker, Kubernetes, CDP (Cloud)

Big Data Tools (Hive, Impala, HBase)

Programming (Python / Scala / Shell)

Data Storage & File Formats (Parquet, Avro, ORC, JSON)

Cluster Management & Performance Tuning

Data Ingestion Tools (Sqoop, Flume, Kafka)

Experience with real-time processing (Spark Streaming / Kafka)

Knowledge of data governance and security frameworks

Exposure to cloud platforms (Azure / AWS / GCP) * Basic understanding of data warehousing concepts, Cloudera Certified Associate (CCA) Data Analyst/ Cloudera Certified Professional (CCP) Data Engineer (highly preferred) Cloudera Certified Administrator (CCA Admin)