Senior Data Engineer
Role details
Job location
Tech stack
Job description
Pipeline Development: Build and maintain scalable, robust ETL/ELT pipelines to ingest, clean, and transform data from diverse sources.
Infrastructure Management: Design, create, and maintain data warehouses, data lakes, and database architectures.
Data Optimization: Enhance data flow, collection, and storage to ensure data quality and reliability.
System Integration: Integrate data products into existing systems and business processes.
Collaboration: Work with data scientists and analysts to support machine learning, modelling, and reporting initiatives.
Security & Compliance: Implement data security, privacy policies, and best practices for data governance.
Requirements
Do you have experience in Spark?, Do you have a Bachelor's degree?, Technical Proficiency: Strong expertise in SQL and programming languages, specifically Python, Scala, or Java.
Data Technologies: Experience with big data tools and frameworks (e.g., Spark, Kafka, Hadoop).
Cloud Platforms: Proficiency in cloud services (e.g., AWS, Azure, GCP).
Data Modeling: Understanding of database design, normalization, and data modeling patterns.
Database Management: Experience with relational and NoSQL databases.
Education: Typically requires a bachelor's degree in Computer Science, Engineering, or a related field.
Core Competencies
Problem-Solving: Strong analytical skills to diagnose data quality issues and optimize system performance.
Communication: Ability to explain technical concepts to non-technical stakeholders.
Agile Development: Experience with DevOps practices and iterative, fast-paced development.