Data Engineer

TechniPros, LLC

Nashville, TN, United States

about 1 month ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

dice.com

Tech stack

Java (Programming Language) Airflow Amazon Web Services Apache HTTP Server Microsoft Azure Big Data BigQuery Cloud Database Continuous Integration Data Architecture Information Engineering Data Governance

+34 more

Data Integration Extract Transform Load (ETL) Data Systems Data Warehousing Database Design Dimensional Modeling Distributed Computing Environment Python (Programming Language) Query Optimization SQL Stored Procedures SQL Databases Unstructured Data Enterprise Data Management Google Cloud Cloud Platform System Real Time Systems Data Ingestion Azure Data Factory Snowflake Apache Spark Database Performance Infrastructure as Code (IaC) Data Lakes Kubernetes AWS Glue Apache Kafka Video Streaming Terraform Stream Processing Azure Synapse Analytics Data Pipelines Docker Amazon Redshift Databricks

Job description

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and enterprise data platforms. The ideal candidate will have strong expertise in cloud-based data engineering, ETL/ELT processing, data warehousing, big data technologies, and modern data architectures. The candidate will collaborate with cross-functional teams to build reliable, secure, and high-performing data solutions., * Design, develop, and maintain scalable ETL/ELT data pipelines.

Build and optimize data ingestion frameworks for structured and unstructured data.
Develop and maintain cloud-based data platforms and data lakes.
Design and implement dimensional data models and data warehouse solutions.
Work with large datasets using distributed computing frameworks.
Develop data integration solutions using batch and real-time processing techniques.
Optimize SQL queries, stored procedures, and database performance.
Implement data quality, governance, security, and compliance controls.
Collaborate with data scientists, analysts, and business stakeholders.
Build CI/CD pipelines for data engineering workflows.
Monitor, troubleshoot, and optimize production data pipelines.
Create technical documentation and maintain operational procedures.

Requirements

Strong experience with Python and/or Java programming.
Extensive experience with SQL, query optimization, and database design.
Hands-on experience with Apache Spark and distributed data processing.
Strong experience with ETL/ELT tools and frameworks.
Experience with AWS/Azure/Google Cloud Platform cloud platforms.
Hands-on experience with Data Warehousing concepts and dimensional modeling.
Experience with Apache Kafka or other streaming technologies.
Experience with Databricks and big data ecosystems.
Strong knowledge of Snowflake, Redshift, Big Query, or similar platforms.
Experience with Airflow, orchestration, and workflow automation.
Knowledge of Docker, Kubernetes, and CI/CD pipelines.
Excellent communication and stakeholder management skills.

Preferred Skills:

Experience with Azure Data Factory (ADF), Azure Synapse, or AWS Glue.
Experience with Delta Lake, Apache Iceberg, or Apache Hudi.
Knowledge of Terraform and Infrastructure as Code (IaC).
Experience with real-time data streaming architectures.
Experience with data governance, lineage, and cataloging tools.
Experience in financial services, healthcare, government, or regulated industries.
Familiarity with Agile/Scrum methodologies.