Data Engineer

We are seeking an experienced Transformation Speed Layer Engineer to join our data engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data pipelines and architectures that enable fast and reliable data transformation and processing.Responsibilities

• Design, develop, and maintain data pipelines and architectures that enable fast data transformation and processing.
• Collaborate with cross-functional teams to identify and prioritize data transformation requirements.
• Ensure data quality, integrity, and security throughout the data transformation process.
• Work closely with data scientists and analysts to develop and optimize data models and algorithms.
• Optimize the performance, scalability, and reliability of data pipelines.
• Develop and maintain technical documentation for data pipelines and architectures.

• Education: Bachelor's degree in Computer Science, Engineering, or a related field.

Technical Skills:

• Programming Languages: Pyspark and Java.
• Data Modeling, Data Ingestion, and Data Warehouse experience.
• Real-time Services: Kafka, Spark Streaming, Flink.
• Distributed Processing: Hadoop, HDFS, Cloudera, Hive.
• Experience with Azure platform.
• Experience with data processing technologies like Apache Beam, Apache Spark, or similar.
• Familiarity with cloud-based platforms such as Google Cloud, Amazon Web Services (AWS), or Microsoft Azure.
• Experience with containerization technologies.
• Understanding of machine learning and data science concepts is a plus.