Data Engineer

• Build and maintain scalable data pipelines for BigQuery using Cloud Composer and Airflow
• Provide observability and monitoring using Monte Carlo and Looker, as well as operational tools such as New Relic and Splunk, to drive reliability, data quality, and robustness
• Build critical data quality checks to ensure the data we process and provide to partner teams meets security and compliance requirements
• Participate in code and technical design reviews
• Scope and plan work with product engineering, data science, and the business
• Support other teams with data tracking, collection, ingestion, quality monitoring, and alerting

Technologies:

• Airflow
• BigQuery
• CI/CD
• Cloud
• Composer
• Docker
• Flink
• GCP
• Support
• Java
• Kubernetes
• Looker
• Python
• SQL
• Scala
• Security
• Spark
• Splunk
• Terraform
• dbt
• AI, We are a leading global SaaS organisation with an International Engineering team, and we offer the opportunity to contribute to an engineering-focused environment while expanding your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform, and Google Cloud Platform. This is a 12-month hybrid contract role with the potential to become permanent. We also offer hybrid working, a great location with fantastic offices, and an award-winning work environment.

• 3+ years of experience managing batch and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, or dbt
• Working knowledge of Python, SQL, and shell scripting
• Experience with Docker/Kubernetes and infrastructure-as-code tools such as Terraform, Salt, or similar
• 3+ years of significant professional experience developing applications in a modern programming environment, preferably Scala, Java, and/or Python
• Professional experience across the entire software development lifecycle, from gathering requirements to production deployment using CI/CD
• Confidence in articulating complex technical content to peers and partners across the organization via technical design documents