Big Data Architect

Gigatec Llc

Annapolis Junction, MD, United States

2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

careerjet.com

Tech stack

Java (Programming Language) Airflow Amazon Web Services Confluence JIRA Bash Shell Big Data Information Engineering Linux Python (Programming Language) Software Engineering Delivery Pipeline

+6 more

Apache Spark Git Containerization Atlassian Tools Software Version Control Docker

Job description

What You’ll Be Doing (We don’t love the bullet points, but we love the work!) In this key Software Engineer role, you’ll be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow, keeping critical processes flowing as smoothly as a well-timed morning coffee. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases, because no one wants a pipeline that is more decaf than dependable. You’ll get to collaborate with data engineers, platform teams, and stakeholders to ensure workflows are efficient, secure, and production ready, delivering results with the consistency of a favorite espresso order.

Requirements

Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Recent software development experience using Python and Java
Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
Experience using Distributed Big Data processing engines including Apache Spark
Experience with containerization technologies such as Docker, containerd, and Podman
Experience with Git Source Control System, * Familiar with AWS Cloud Services and Infrastructure
Experience using the Atlassian Tool Suite (JIRA, Confluence)
Fluent in sarcasm, kindness, and the occasional office banter