Data Engineer
Role details
Job location
Tech stack
Job description
-
Design, build, and maintain highly scalable and reliable data pipelines to move and transform data across large-scale, distributed systems at exabyte scale.
-
Architect, implement, and deploy new data models and data processing workflows in production, ensuring data quality, integrity, and compliance with data privacy regulations.
-
Collaborate with cross-functional teams - including software engineers, data scientists, and product managers - to understand data requirements and deliver effective, high-impact solutions.
-
Develop and optimize data-driven systems and solutions to enhance operational scale, efficiency, and generate actionable business insights aligned with evolving product and business needs.
-
Work closely with engineering teams to plan and execute on half-yearly roadmaps that align with business priorities and available engineering capacity.
-
Perform data quality monitoring, root cause analysis, and proactive resolution of data pipeline issues to ensure high reliability and trust in data assets.
-
Develop comprehensive documentation for all data pipelines, models, and related processes, and cross-train team members on new and evolving data systems.
Requirements
-
- 4+ years of experience in a data engineering role, with a proven track record of designing and managing large-scale data pipelines in a production environment.
Expert-level proficiency in SQL (Presto SQL and/or Spark SQL) and strong programming skills in Python, Java, or Scala.
-
Hands-on experience with big data technologies including Apache Spark, Presto, and Hive, including query optimization and performance tuning at scale. * Ability to work independently and collaboratively in a fast-paced, ambiguous environment with a strong bias for action and impact.
-
Excellent communication skills with the ability to articulate complex technical concepts clearly to both technical and non-technical stakeholders across engineering, product, and business teams.
-
Strong problem-solving and analytical skills, with a deep understanding of business drivers and a focus on delivering measurable impact.
-
Experience with real-time data processing and streaming technologies (e.g., Apache Kafka, Flink, or Scuba-equivalent systems).
-
Demonstrated experience with data privacy, data governance, and compliance frameworks in a large-scale enterprise environment.
-
Excellent communication and interpersonal skills, with the ability to communicate complex technical concepts to both technical and non-technical audiences.
-
Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.