Bigdata Engineer with PySpark

We are seeking a highly skilled Senior Data Engineer with a strong background in Big Data technologies, ETL processes, and hands-on experience with AWS. The ideal candidate wit be responsible for designing, developing, and maintaining robust data pipelines and architectures that support our data-driven decision-making processes. This role requires a deep understanding of data engineering principles and the ability to work collaboratively in a fast-paced environment. Responsibilities:

• Design, develop, and implement scalable data pipelines and ETL processes to ingest, process, and store large volumes of data.
• Utilize Big Data technologies to manage and analyze complex datasets, ensuring data quality and integrity.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
• Optimize data storage and retrieval processes to enhance performance and reduce costs.
• Implement data governance and security measures to protect sensitive information.
• Monitor and troubleshoot data pipeline performance, making necessary adjustments to improve efficiency.
• Stay current with industry trends and emerging technologies in data engineering and Bajt Data

• Strong expertise in Big Data technologies such as Hadoop, Spark, or Kafka.
• Hands-on experience with ETL tools and processes.
• Proficient in AWS services related to data engineering, including but not limited to S3, Redshift, Glue, and EMR.
• Solid programming skills in languages such as Python, Java, or Scala.
• Experience with data modeling and database design.
• Strong analytical and problem-solving skills.

Preferred Skills:

• Familiarity with data visualization tools such as Tableau or Power Bl.
• Experience with containerization technologies like Docker or Kubernetes.
• Knowledge of machine learning concepts and frameworks.
• Understanding of data privacy regulations and compliance standards.