Data Engineer

Agile Resources
Orlando, FL, United States
about 1 month ago

Role details

Contract type
Temporary to permanent
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Compensation
$124,800.0 - $145,600.0
Working hours
Regular working hours

Tech stack

Cloud Computing Continuous Delivery Continuous Integration Information Engineering Python (Programming Language) SQL Databases Large Language Models Apache Spark Event Driven Architecture Apache Kafka Machine Learning Operations

Job description

We are partnering with an industry leader in the construction supplies and equipment space to find a Sr. Data Engineer to spearhead the evolution of their global, next-generation data ecosystem. If you thrive on solving intricate, massive-scale data puzzles, optimizing bleeding-edge distributed computing environments, and laying the groundwork for sophisticated machine learning and LLM operations, this is your next definitive career move. In this role, you will act as the architect behind a mission-critical platform, transforming raw datasets into powerful, secure, and production-ready intelligence that fuels executive-level decisions. You will enjoy a high degree of technical ownership, bridging the gap between advanced cloud engineering and real-world AI applications.

Here’s what you’ll be doing:

  • Architect and optimize high-throughput, distributed computing frameworks and modern Lakehouse structures to handle massive data velocity and volume.
  • Construct robust, production-grade features and pipelines that seamlessly operationalize large language models (LLMs) and predictive machine learning models.
  • Build automated ingestion frameworks across multi-cloud environments, utilizing both real-time streaming and scheduled batch processing.
  • Establish solid data quality, unified cataloging, and access controls while aggressively optimizing cluster performance and cloud infrastructure costs.
  • Collaborate with cross-functional leadership, finance, and operations teams to convert complex strategic goals into highly scalable technical solutions.

Requirements

  • 7+ years of sophisticated data engineering experience, highlighted by mastery-level knowledge of Spark tuning, partitioning, and cloud infrastructure.
  • Extensive hands-on experience designing secure, ACID-compliant storage layers, ideally utilizing modern unified cataloging tools.
  • Expertise in Python and SQL, coupled with practical experience supporting ML lifecycle management (such as tracking, feature stores, or LLM integrations).
  • A proven track record of integrating disparate, complex data sources and maintaining high-availability production environments at enterprise-scale.
  • Familiarity with event-driven architectures (like Kafka), continuous integration/continuous deployment (CI/CD) pipelines, and infrastructure-as-code will be a plus.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on jobs.gotoagile.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:34 min

Capabilities of the Apache Spark processing engine

Ayon Roy Ā· LIVE

1:37 min

Introduction to Apache Kafka benchmarking and performance analysis

Kirill Kulikov Ā· LIVE

1:14 min

Evolution of distributed SQL database architectures

Wei Hu Wei Hu Ā· WWC 2024

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou Ā· Coffee With Developers

1:41 min

Visualizing the complex developer journey for JVM ecosystems

Bobur Umurzokov Ā· LIVE

2:04 min

Comparing offline data analytics with online stream processing

Artem Volk Artem Volk +1 Ā· WWC 2024

Videos

See all

Related articles

See all