Data Engineer
Ventechsoft Inc
Dallas, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours
Job source
Tech stack
Agile Methodology
Airflow
Amazon Web Services
Microsoft Azure
Business Intelligence Development
BigQuery
Cloud Computing Security
Continuous Integration
Data Architecture
Information Engineering
Data Governance
Extract Transform Load (ETL)
+32 more
Data Security
Data Systems
Data Warehousing
DevOps
Distributed Data Store
Python (Programming Language)
NoSQL
Operational Databases
Scrum Methodology
Cloud Services
SQL Databases
Unstructured Data
Data Processing
Google Cloud
Cloud Platform System
Data Ingestion
Snowflake
Database Optimization
Apache Spark
Git
Data Lakes
Pyspark
Kubernetes
Apache Flink
Apache Kafka
Video Streaming
Stream Processing
Azure Synapse Analytics
Data Pipelines
Docker
Amazon Redshift
Databricks
Job description
We are looking for an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data processing solutions. The ideal candidate will have strong experience with cloud data platforms, ETL/ELT, SQL, Python, and modern data engineering technologies., * Design, develop, and maintain scalable ETL/ELT data pipelines.
- Build and optimize data ingestion and transformation processes.
- Develop data solutions using Python, SQL, and PySpark.
- Work with structured and unstructured data from multiple sources.
- Implement data quality, validation, monitoring, and error-handling processes.
- Develop and optimize data models for analytics and reporting.
- Work with cloud-based data platforms and storage solutions.
- Collaborate with Data Scientists, BI Developers, Software Engineers, and business teams.
- Optimize queries, pipelines, and data processing jobs for performance and scalability.
- Implement data security, governance, and access-control best practices.
- Troubleshoot production data pipeline issues and provide root-cause analysis.
- Participate in Agile/Scrum ceremonies and contribute to technical documentation.
Requirements
- 5+ years of experience in Data Engineering.
- Strong experience with Python and SQL.
- Experience developing ETL/ELT pipelines.
- Strong knowledge of Apache Spark / PySpark.
- Experience with AWS, Azure, or Google Cloud Platform.
- Experience with data warehouses such as Snowflake, Databricks, Redshift, BigQuery, or Azure Synapse.
- Experience with Apache Kafka or other messaging/streaming technologies.
- Knowledge of relational and NoSQL databases.
- Experience with Git, CI/CD, and DevOps practices.
- Strong understanding of data modeling, data architecture, and database performance tuning.
Preferred Skills
- Experience with Databricks, Delta Lake, or Apache Airflow.
- Experience with Kafka, Flink, or real-time data processing.
- Knowledge of dbt and modern data stack technologies.
- Experience with Docker/Kubernetes.
- Experience working with large-scale distributed data systems.
- Knowledge of data governance and cloud security.
- Experience in Agile development environments.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦