Data Engineer

Ventechsoft Inc

Dallas, United States

3 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Application Programming Interfaces (APIs) Amazon Web Services Microsoft Azure Business Intelligence Development BigQuery Databases Continuous Integration Information Engineering Extract Transform Load (ETL) Data Transformation Data Security Data Warehousing

+22 more

Distributed Computing Environment Python (Programming Language) Cloud Services SQL Databases Data Streaming Data Processing Google Cloud Snowflake Apache Spark Git Data Lakes Pyspark Kubernetes Real Time Data Apache Kafka Data Management Video Streaming Stream Processing Data Pipelines Docker Amazon Redshift Databricks

Job description

We are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions. The ideal candidate will have strong experience with cloud platforms, ETL/ELT, data warehousing, and distributed data processing., * Design, develop, and maintain robust ETL/ELT data pipelines for batch and real-time data processing.

Develop data engineering solutions using Python, SQL, Spark/PySpark, and related technologies.
Work with AWS, Azure, or Google Cloud Platform cloud services to build scalable data platforms.
Develop and optimize data lakes, data warehouses, and data pipelines.
Integrate data from databases, APIs, flat files, and streaming platforms.
Work with Kafka or other messaging/streaming technologies for real-time data ingestion.
Perform data transformation, cleansing, validation, and quality checks.
Optimize SQL queries, data pipelines, and processing jobs for performance and scalability.
Implement data security, governance, monitoring, and reliability best practices.
Collaborate with Data Scientists, BI Developers, Software Engineers, and business stakeholders.

Requirements

8+ years of hands-on Data Engineering experience.
Strong Python and SQL programming skills.
Experience with Apache Spark / PySpark.
Strong understanding of ETL/ELT, data modeling, and data warehousing.
Experience with AWS/Azure/Google Cloud Platform cloud platforms.
Experience with Kafka or real-time streaming technologies.
Knowledge of Data Lakes, Databricks, Snowflake, Redshift, BigQuery, or similar platforms.
Experience with Git, CI/CD, Docker, and Kubernetes is a plus.
Strong analytical, problem-solving, and communication skills.