Data Engineer - Hybrid

As a Data Engineer, you will support the design, development, and maintenance of enterprise data platforms and large-scale data processing solutions. You will be responsible for building and optimizing data pipelines, developing ETL processes, and ensuring the availability, reliability, and quality of data across business-critical systems. This role requires expertise in big data technologies, SQL optimization, and distributed data processing, along with the ability to collaborate with cross-functional teams to deliver scalable and efficient data solutions., * Design, implement, and maintain enterprise ETL processes and data pipelines

• Develop scalable and efficient code to process, transform, and deliver large datasets
• Build and optimize data pipelines using Apache Spark, Hadoop, and related big data technologies
• Collaborate with engineering and analytics teams to solve complex data challenges and maintain data quality
• Support the delivery of accurate and actionable data solutions for business stakeholders
• Design and manage distributed data processing workflows and orchestration processes
• Develop and optimize SQL queries for large-scale data retrieval, transformation, and analysis
• Participate in data modeling and database design initiatives to support scalable solutions
• Monitor, troubleshoot, and resolve data processing and pipeline issues
• Automate routine data management tasks and improve operational efficiency
• Apply testing and validation practices to ensure data accuracy, consistency, and reliability
• Participate in code reviews and follow development best practices and version control standards
• Build strong working relationships with internal teams and business stakeholders
• Ensure compliance with organizational policies, standards, and regulatory requirements

• Experience as a Data Engineer or in a similar data-focused engineering role
• Strong expertise in writing and optimizing SQL queries for large datasets
• Hands-on experience with Apache Spark, including PySpark, Spark SQL, and Spark Streaming
• Experience working with Hadoop ecosystem technologies such as HDFS, Hive, and YARN
• Strong understanding of ETL frameworks and data pipeline development
• Knowledge of distributed data processing and big data architectures
• Understanding of data modeling concepts and database design principles
• Experience working with Python for data engineering and automation tasks
• Ability to analyze, troubleshoot, and resolve complex data issues independently
• Knowledge of data testing, validation, and quality assurance practices
• Strong verbal and written communication skills with the ability to collaborate with technical and non-technical stakeholders
• Experience with version control, code reviews, and software development best practices
• Ability to work effectively in a collaborative, fast-paced environment
• Bachelor's degree in Engineering, Mathematics, Finance, Business, Computer Science, or a related quantitative field, or equivalent practical experience