Data Engineer

TechniPros, LLC
Nashville, United States of America
11 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Remote
Nashville, United States of America

Tech stack

Java
Airflow
Amazon Web Services (AWS)
Apache HTTP Server
Azure
Big Data
Google BigQuery
Cloud Database
Continuous Integration
Data Architecture
Information Engineering
Data Governance
Data Integration
ETL
Data Systems
Data Warehousing
Database Design
Dimensional Modeling
Distributed Computing Environment
Python
Query Optimization
SQL Stored Procedures
SQL Databases
Unstructured Data
Enterprise Data Management
Google Cloud Platform
Cloud Platform System
Real Time Systems
Data Ingestion
Azure
Snowflake
Spark
Database Performance
Infrastructure as Code (IaC)
Data Lake
Kubernetes
Amazon Web Services (AWS)
Kafka
Video Streaming
Terraform
Stream Processing
Azure
Data Pipelines
Docker
Redshift
Databricks

Job description

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and enterprise data platforms. The ideal candidate will have strong expertise in cloud-based data engineering, ETL/ELT processing, data warehousing, big data technologies, and modern data architectures. The candidate will collaborate with cross-functional teams to build reliable, secure, and high-performing data solutions., * Design, develop, and maintain scalable ETL/ELT data pipelines.

  • Build and optimize data ingestion frameworks for structured and unstructured data.
  • Develop and maintain cloud-based data platforms and data lakes.
  • Design and implement dimensional data models and data warehouse solutions.
  • Work with large datasets using distributed computing frameworks.
  • Develop data integration solutions using batch and real-time processing techniques.
  • Optimize SQL queries, stored procedures, and database performance.
  • Implement data quality, governance, security, and compliance controls.
  • Collaborate with data scientists, analysts, and business stakeholders.
  • Build CI/CD pipelines for data engineering workflows.
  • Monitor, troubleshoot, and optimize production data pipelines.
  • Create technical documentation and maintain operational procedures.

Requirements

  • Strong experience with Python and/or Java programming.
  • Extensive experience with SQL, query optimization, and database design.
  • Hands-on experience with Apache Spark and distributed data processing.
  • Strong experience with ETL/ELT tools and frameworks.
  • Experience with AWS/Azure/Google Cloud Platform cloud platforms.
  • Hands-on experience with Data Warehousing concepts and dimensional modeling.
  • Experience with Apache Kafka or other streaming technologies.
  • Experience with Databricks and big data ecosystems.
  • Strong knowledge of Snowflake, Redshift, Big Query, or similar platforms.
  • Experience with Airflow, orchestration, and workflow automation.
  • Knowledge of Docker, Kubernetes, and CI/CD pipelines.
  • Excellent communication and stakeholder management skills.

Preferred Skills:

  • Experience with Azure Data Factory (ADF), Azure Synapse, or AWS Glue.
  • Experience with Delta Lake, Apache Iceberg, or Apache Hudi.
  • Knowledge of Terraform and Infrastructure as Code (IaC).
  • Experience with real-time data streaming architectures.
  • Experience with data governance, lineage, and cataloging tools.
  • Experience in financial services, healthcare, government, or regulated industries.
  • Familiarity with Agile/Scrum methodologies.

Apply for this position