Data Engineer

We are looking for a skilled Data Engineer to join our team in Houston, TX. The ideal candidate will have strong experience in PySpark, Microsoft Azure, Azure Data Factory (ADF), SQL, and Data Warehousing. This role focuses on designing scalable data pipelines, building enterprise ETL/ELT solutions, and delivering high-quality data integration across cloud-based platforms., * Design, develop, and optimize scalable data pipelines.

• Build, maintain, and enhance ETL/ELT workflows using Azure Data Factory (ADF).
• Process, transform, and optimize large datasets using PySpark.
• Develop efficient data integration solutions across multiple enterprise systems.
• Create scalable and reliable cloud-based data engineering solutions on Microsoft Azure.
• Optimize SQL queries, data warehouse performance, and data processing workflows.
• Ensure data quality, reliability, consistency, and governance.
• Troubleshoot production issues and optimize pipeline performance.
• Collaborate with cross-functional teams including business analysts, data scientists, and application developers.
• Support continuous improvements in data architecture and engineering best practices.

• Strong experience in PySpark.
• Hands-on experience with Microsoft Azure.
• Expertise in Azure Data Factory (ADF).
• Strong knowledge of ETL/ELT development.
• Experience designing scalable data pipelines.
• Strong SQL programming and database optimization skills.
• Experience with Data Warehousing concepts and implementation.
• Experience in data integration and performance tuning.
• Knowledge of cloud-based data engineering best practice.

Preferred Skills:

• Experience with Azure Databricks.
• Knowledge of Delta Lake, Spark SQL, and Azure Synapse Analytics.
• Familiarity with CI/CD pipelines and DevOps practices.
• Experience working in Agile/Scrum environments.