hybrid Data Engineer

Horizontal Talent

Knoxville, TN, United States

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Java (Programming Language) Apache ActiveMQ Airflow Microsoft Azure Big Data BigQuery Cloud Database Computer Programming Databases Information Engineering Data Governance Extract Transform Load (ETL)

+30 more

Apache Hadoop Hadoop Distributed File System Apache HBase Apache Hive Python (Programming Language) PostgreSQL Microsoft SQL Server MongoDB MySQL NoSQL Apache Oozie Oracle (Applications) Performance Tuning RabbitMQ Talend Google Cloud Apache Spark Indexer Storage Technologies Collibra Apache Flink Cassandra Real Time Data Apache Kafka Apache Nifi Spark Streaming Data Management Data Pipelines Amazon Redshift Microservices

Job description

Join a hybrid Data Engineer opportunity focused on building and improving modern big data solutions for ingestion, integration, and transformation. This role supports impactful data initiatives and offers flexibility to work from an East Coast office or on-site in Knoxville, TN when needed. Responsibilities

Design and implement data engineering components that support client requirements and technical solution goals.
Build and maintain batch and real-time data ingestion, integration, and transformation workflows.
Work with large-scale data processing and storage technologies to support reliable, efficient data pipelines.
Contribute to analytics, search, and aggregation capabilities across data platforms.
Participate in technical design discussions and help ensure the overall health and performance of the solution.
Support cloud-based data platform services and optimize data workflows for performance and scalability.
Collaborate with project teams in a hybrid work environment and attend onsite interviews or equipment pickup when required.

Requirements

At least 4 years of hands-on experience with big data technologies.
Experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, NiFi, Airflow, Flink, Storm, Oozie, or similar platforms.
Strong programming experience in Java, Python, or Scala, with Java preferred.
Working knowledge of NoSQL and MPP data platforms such as HBase, MongoDB, Cassandra, AWS Redshift, Azure SQLDW, or Google Cloud Platform BigQuery.
Experience with AWS data platform services.
Ability to translate business needs into technical design and implementation.
Strong problem-solving skills and the ability to work independently on design discussions and solution delivery.

Preferred Skills

Experience with traditional ETL tools such as Informatica or Talend.
Familiarity with database technologies such as Oracle, MySQL, SQL Server, or Postgres.
Knowledge of data governance practices, including security, lineage, and catalog tools like Collibra or Alation.
Exposure to distributed messaging tools such as ActiveMQ, RabbitMQ, or Solace.
Experience with search/indexing and microservices architectures.
Background in performance tuning and data pipeline optimization.
Cloud data certifications or other relevant big data credentials.