Senior Data Engineer (UK)

External Posting
London, UK
3 days ago
Apply on www.adzuna.co.uk
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
£74,846.0
Working hours
Regular working hours

Tech stack

Artificial Intelligence Amazon S3 Data Validation Information Engineering Data Governance Data Infrastructure Data Visualization Dimensional Modeling Python (Programming Language) SQL Azure Operational Databases Raw Data
+12 more
Power BI Azure Data Lake SQL Databases Enterprise Data Management Apache Spark Data Layers Pyspark Production Code Performance Monitor Star Schema Data Pipelines Databricks

Job description

At the Rainforest Alliance, we believe in the value of data collected at field and company level. With the increasing update of technologies, the amount of data produced will lead to an exponential amount and growth of available information. Unlocking the value of the information and linking it to informed decision-making will enable our organization in reaching the goal of a world where human and nature thrive in harmony.

Rainforest Alliance is building a Unified Data & AI-ready Platform on Databricks, Azure Data Lake Storage Gen2 and Power BI Premium, using a Bronze/Silver/Gold medallion architecture.

In order to progress to a singular platform, we are looking to onboard additional data engineering capability to support both the existing Azure SQL based services and the new platform, which is in pilot phase, as part of wider improvements to the hours of service that we provide to our global users - A working knowledge of both technologies is therefore required. In addition, to provide escalation cover and time zone management, this role will be responsible for the supervision of a team of (currently 3) data and data platform engineers.

You will report to the Senior Manager, Enterprise Data Platform.

WHAT YOU WILL DO:

  • Pipeline engineering: Design, build and maintain robust, production-grade data pipelines and core platform components on Databricks / ADLS Gen2, following the Bronze/Silver/Gold medallion architecture.
  • Dimensional modelling: Translate signed-off requirements into dimensional models and the queries that take consolidated raw data to the dimensional model.
  • Platform engineering: Build the generic, reusable functions and production code that reproduce, at scale and across the organisation, the same output as the validated queries; Maintain and continue development of the existing SQL-based environment.
  • Reporting: Support the design, development and enhancement of Power BI datasets, reports and dashboards; Provide support to report builders/owners in reviewing and optimising reports and datasets, to ensure accuracy, mantainability and performance.
  • Power BI Tenant Admin: Support Power BI Premium lifeycle management processes, including user onboarding/offboarding, governance activities (including obsolete workspace and dataset/report cleanup), and performance monitoring/optimisation.
  • Data visualisation enablement: Develop and support the data visualisation methodologies, views and extractions (e.g. Databricks views feeding Power BI).
  • Data quality: Work with the Survey Design Specialists, BI Analysts and M&E Analysts to embed data quality checks and troubleshoot issues found.
  • Documentation & reproducibility: Maintain clear, field-knowledge-free documentation of ingestion, transformation and modelling logic.
  • Collaboration: Work closely with the Data Platform Lead, Tech, and Field Implementation to ensure engineering work stays aligned with the wider UDP architecture and governance model (Unity Catalog).
  • Leadership: Matrix manage a small team (3-5) of data and data platform engineers, be accountable for escalations and ensure smooth transition between regional teams to support a follow-the-sun approach to project delivery and “business as usual” support.

Requirements

  • Experience building and operating production data pipelines, ideally on Databricks (or a comparable Spark-based platform) and cloud data lake storage (ADLS Gen2, S3 or equivalent).
  • SQL and hands-on experience with dimensional modelling (star schema / Bronze-Silver-Gold or similar medallion patterns).
  • Experience building the data layer behind BI tools, ideally Power BI, including designing performant, well-governed semantic models and views.
  • Write clear technical documentation that a non-specialist reader can follow.
  • Comfortable working to formal sign-off gates (requirements, staging, production) and coordinating with analysts, survey and field teams rather than working in isolation.

Desirable:

  • Experience with survey/ODK-style data collection tools (e.g. Survey123, XLSForms).
  • Familiarity with data governance concepts (data catalogues, access control, e.g. Unity Catalog) in a multi-tenant or multi-partner data platform.
  • Python and PySpark for pipeline development and testing.

Benefits & conditions

  • Be a part of a global organization with an impactful mission and a collaborative, respectful, and accountable culture.
  • Enjoy opportunities for professional growth and career development.
  • Benefit from remote working flexibility and flexible hybrid working environment.
  • A favourable time‑off policy.
  • Prioritize your wellbeing. We have numerous ways to promote work-life balance.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.adzuna.co.uk
Prepare application

Good distractions

Loading talks and stories from around this role…