Senior Specialist - Data Engineering

Experteer Overview As a Data Engineer, you will design, develop and maintain scalable data pipelines in on-premise Hadoop environments. You will work with PySpark, HDFS and Hive to process large-scale data and optimize Spark jobs. You’ll troubleshoot data pipelines, enforce data quality practices, and collaborate with cross-functional teams to enable reliable data-driven decisions. This role focuses on hands-on implementation and performance tuning in a fast-paced setting, with exposure to cloud platforms as a plus. Compensation / Benefits * design and maintain scalable data pipelines * develop and optimize Spark jobs (PySpark) * work with HDFS and Hive for data processing and warehousing * data ingestion and integration using Sqoop * troubleshoot data pipeline issues and performance bottlenecks * enforce data quality governance and best practices * support onprem Hadoop ecosystem architectures and tuning * collaborate with stakeholders in fast-paced environments Tasks * 5+ years of experience with HDFS, Hive and Spark * strong hands-on experience in the Hadoop ecosystem (on-premise) * expertise in PySpark for large-scale data processing * experience building and optimizing Spark jobs * hands-on data ingestion experience with Sqoop * good understanding of distributed data processing and big data concepts * ability to design, develop and maintain scalable data pipelines * experience with large volumes of structured and unstructured data * strong problem-solving skills and independence in fast-paced environments * exposure to cloud platforms (AWS/Azure/GCP) is a plus Key requirements *

_ of experience with HDFS, Hive and Spark * strong hands-on experience in the Hadoop ecosystem (on-premise) * expertise in PySpark for large-scale data processing * experience building and optimizing Spark jobs * hands-on data ingestion experience with Sqoop * good understanding of distributed data processing and big data concepts * ability to design, develop and maintain scalable data pipelines * experience with large volumes of structured and unstructured data * strong problem-solving skills and independence in fast-paced environments * exposure to cloud platforms (AWS/Azure/GCP) is a plus Key requirements *