Data Engineer

Horizontal Talent

Arlington, VA, United States

22 days ago

Apply on www.indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.indeed.com

Tech stack

Java (Programming Language) Apache ActiveMQ Adobe InDesign Airflow Big Data BigQuery Cloud Database Computer Programming Databases Information Engineering Data Governance Data Infrastructure

+31 more

Data Integration Extract Transform Load (ETL) Apache Hadoop Hadoop Distributed File System Apache HBase Apache Hive Python (Programming Language) PostgreSQL Messaging Application Programming Interface Microsoft SQL Server SQL Azure MongoDB MySQL NoSQL Oracle (Applications) Performance Tuning RabbitMQ Talend Cloud Platform System Apache Spark Indexer Collibra Apache Flink Cassandra Apache Kafka Apache Nifi Spark Streaming Data Management Data Pipelines Amazon Redshift Microservices

Job description

Design, build, and support data pipelines that handle batch and real-time ingestion from multiple sources
Develop solutions for data integration, transformation, storage, search, and aggregation
Work across big data frameworks and platforms to improve scalability, reliability, and performance
Collaborate with technical teams to translate business needs into practical data engineering solutions
Participate in design discussions and help ensure the overall health of the solution architecture
Support data engineering efforts involving cloud-based infrastructure and analytics use cases
Contribute to troubleshooting, optimization, and enhancement of existing data workflows

Requirements

4+ years of relevant experience working with big data technologies
Hands-on experience with Hadoop ecosystem tools such as HDFS, Kafka, Spark, Spark Streaming, Hive, Airflow, NiFi, Flink, or similar technologies
Strong programming skills in Java, Scala, or Python, with Java preferred
Experience working with NoSQL and MPP data platforms such as HBase, MongoDB, Cassandra, Redshift, Azure SQL DW, or BigQuery
Working knowledge of AWS data platform services
Ability to design and implement data engineering solutions independently
Strong problem-solving, communication, and collaboration skills
Bachelor’s degree or equivalent combination of education, training, and experience, * Experience with ETL tools such as Informatica or Talend
Familiarity with databases such as Oracle, MySQL, SQL Server, or Postgres
Knowledge of data governance practices including security, lineage, and cataloging
Exposure to tools such as Collibra or Alation
Understanding of messaging frameworks such as ActiveMQ, RabbitMQ, or Solace
Experience with search and indexing technologies
Background in microservices architecture
Experience with performance tuning and pipeline optimization
Cloud data certifications or other relevant big data credentials

Horizontal is committed to fostering an inclusive and welcoming environment where different perspectives are valued and every team member can contribute meaningfully. We believe diversity, equity, and inclusion strengthen our teams and help drive better outcomes for everyone.