Lead Data Engineer (Full-Time, Direct Hire)

The Jupiter Group
Houston, United States of America
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Houston, United States of America

Tech stack

Amazon Web Services (AWS)
Data analysis
Big Data
Cloud Database
Information Systems
Data Governance
Data Infrastructure
ETL
Dataspaces
Data Systems
Relational Databases
Programming Tools
Distributed Data Store
Python
PostgreSQL
Microsoft Visual Studio
Microsoft SQL Server
SQL Azure
Oracle Applications
Power BI
Teradata
Enterprise Data Management
Data Processing
Cloud Platform System
GIT
Powerquery
Pandas
PySpark
Information Technology
Integration Frameworks
SQL Server Management Studio (SSMS)
Software Version Control
Data Pipelines
Databricks

Job description

The Lead Data Engineer will provide technical leadership in designing, developing, and optimizing modern enterprise data platforms. This individual will drive the implementation of cloud-based data solutions, establish scalable data frameworks, and improve the reliability, accessibility, and governance of enterprise data assets. The role partners closely with business stakeholders, IT teams, and leadership to deliver solutions that support critical operations across a growing midstream organization.

Requirements

  • 8+ years of experience designing and delivering enterprise data engineering solutions, including significant experience with cloud-based platforms.
  • At least 5 years of hands-on experience building data solutions within Amazon Web Services (AWS).
  • Strong experience working with Databricks for large-scale data processing and analytics.
  • Experience developing and maintaining ETL/ELT pipelines that support enterprise reporting and analytics.
  • Advanced SQL development skills along with strong Python programming experience, including data processing libraries such as PySpark and Pandas.
  • Experience with reporting and analytical technologies including Power BI, DAX, and Power Query.
  • Familiarity with enterprise data cataloging and governance platforms.
  • Experience working with relational database platforms including SQL Server, Oracle, PostgreSQL, Azure SQL, and Teradata.
  • Knowledge of source control and development tools such as Git, Visual Studio Code, and SQL Server Management Studio.
  • Understanding of modern data architecture concepts including cloud data lakes, Lakehouse environments, distributed data models, and service-oriented integration.
  • Strong communication skills with the ability to collaborate across technical and business teams.
  • Demonstrated leadership experience guiding technical initiatives and mentoring engineers.

Preferred Experience

  • AWS and/or Databricks certifications.
  • Experience modernizing legacy data environments and migrating enterprise workloads to cloud platforms.
  • Background designing secure, scalable, and high-performing data ecosystems.
  • Experience implementing enterprise data governance, security, and compliance standards.
  • Proven ability to lead technical projects from planning through implementation.
  • Experience partnering with business leaders to deliver data-driven solutions that improve operational performance and decision-making.
  • Prior experience supporting organizations within the energy, oil & gas, or midstream sector is highly valued.

Education

Required

  • Bachelor''''s degree in Computer Science, Information Systems, Engineering, Data Science, or a related technical discipline.

About the company

Our client, a well-established company in the midstream industry, is seeking an experienced Lead Data Engineer to join its technology team in Houston. This is a full-time, direct-hire opportunity offering the chance to help shape the organization''''s enterprise data strategy while building scalable solutions that support business intelligence, operational reporting, advanced analytics, and emerging AI initiatives. **This company is unable to provide sponsorship now or at any point in the future for applicants**

Apply for this position