Senior Data Engineer - AWS/PySpark/Snowflake/DBT

The successful candidate will have strong hands-on experience building and supporting production-grade data pipelines using AWS, PySpark, Snowflake, SQL and DBT, combined with solid software engineering and DevOps practices., * Design, build and maintain production-grade data pipelines and services across AWS, PySpark, Snowflake and SQL-based transformation frameworks

• Develop modular, testable and maintainable data models and transformation pipelines using DBT or equivalent SQL transformation frameworks
• Build and optimise large-scale data processing solutions
• Improve pipeline reliability through automation, monitoring, validation and dependency management
• Diagnose and resolve production pipeline failures and data quality issues
• Manage pipeline reruns, validation and recovery processes for critical data services
• Optimise Snowflake queries and data models for performance and scalability
• Contribute to CI/CD and DevOps practices using tools such as GitHub Actions
• Apply strong software engineering practices including testing, modular development, version control and SDLC standards
• Contribute to technical design, architecture and solutioning discussions
• Work closely with cross-functional data, engineering and product teams

Candidates should have strong commercial experience with:

• AWS data stack - S3, EMR, EC2
• PySpark
• Snowflake
• DBT
• Advanced SQL
• Production-grade data pipelines
• Large-scale data processing
• Data modelling
• RDV/BDV or similar data modelling patterns
• Snowflake and query performance optimisation
• Modular and testable data transformation development
• Structured SQL transformation frameworks
• GitHub Actions
• CI/CD
• Git/version control
• Strong software engineering and SDLC practices

Experience or exposure to Infrastructure-as-Code tools such as Terraform is also highly desirable.

Nice to Have

• Exposure to MLOps
• Experience supporting data science or machine-learning pipelines
• Experience working within engineering-led data or product teams