Azure Data Engineer

You’ll report to the Lead Data Engineer and be part of our Data Engineering Team, working closely with architects, analysts, testers and delivery squads in an agile, cross-functional setup. You’ll be supported to learn, contribute, and help shape the platform we’re building., * Build and maintain scalable, automated data pipelines using Azure tools like Data Factory, Databricks, and Logic Apps etc.

• Ingest, transform and model data from multiple sources into our enterprise data platform.
• Optimise data flows for performance, reliability, and reusability - with a driven, analytical, and decisive mindset.
• Support data quality, reconciliation and monitoring processes.
• Work autonomously across the full data lifecycle - from raw source to insight - showing initiative and self-reliance.

Key relationships.

• Data Engineering Team - Collaborate on design, delivery and best practices.
• Solution Architects - Align pipeline design with platform standards and architecture.
• Product Owners - Understand business needs and translate them into technical data solutions.
• Test Analysts - Ensure the integrity and reliability of data delivered to end users.
• Reporting & BI Teams - Provide clean, structured data for dashboards and analytics.

Your engineering work will directly support the department’s goal of delivering clean, connected, and reliable data for the organisation. By automating pipelines, reducing manual effort, and improving data quality, you’ll help speed up delivery and reduce risk. Close collaboration with architects, analysts, and reporting teams ensures we’re building the right things - the right way - to meet evolving business needs, support self-service reporting, and grow our cloud-first data platform.

• Relevant degree or equivalent experience in IT, Computer Science, Data Engineering, or a related field (preferred but not mandatory)., * Experience with databases and analytics, including relational databases (e.g. SQL Server), Data Warehousing, Big Data and business analytics using visualisation tools (Power BI).
• Experience designing solution using Microsoft Azure stacks such as Azure Data Lake, Azure Data Factory, Logic Apps, Azure Data Bricks, Azure functions, Azure Synapse.
• Must have previously worked in Data Warehouse ETL & BI platforms, with a good understanding of DW strategy, architecture, data modelling, schemas following Kimball Methodology.
• Good understanding on Master Data Services, DQ profiling, cleansing, update, DQ reports, Purview.
• Strong experience with at least one of the programming languages (Python, Scala, PySpark).
• Knowledge and experience using the Microsoft technology stack and associated development tools.
• Experience working with SharePoint, PowerPivot, PowerBI and deep knowledge of the underlying technology to develop data integration layers and associated systems to provide repositories of information.
• Experience working with database, storage, collection and aggregation models, delta loading, external data load.

Desirable:

• Exposure to PowerShell Scripting would be advantageous.
• Experience of working in an agile cross-functional delivery squad working with product owners, solution analysts, solution architects and test analysts.
• Awareness and understanding of key DevOps and cloud concepts including CI/CD, Iaas, PaaS and SaaS with enabled multi-disciplined, cross functional capabilities. Containerisation experience highly desirable.
• Experienced in designing and coordinating existing pilots, prototypes, or proof of concepts.
• Experience of the underlying technology to develop data integration layers and associated systems to provide repositories of information.
• Exposure to Semarchy (MDM Solution) would be advantageous.
• Exposure to NEC Housing Management Solution and PL/SQL would be advantageous.
• Exposure to Total Mobile.

Benefits include: Excellent pension plan (up to 6% double contribution), 28 days Annual Leave rising to 31 days with length of service + Bank Holidays, Westfield Health Cash Plan, non-contributory life assurance, up to 21 hours volunteering paid days, lifestyle benefits, Employee Assistance Programme and many more

L&Q is one of the UK’s largest housing associations and providers of social and affordable homes. We house around 250,000 people primarily across Greater London and Greater Manchester. Our purpose is simple - to provide social homes and landlord services that everyone can be proud of.

We’re building a modern data platform on Azure - and this role puts you right in the action. You’ll get hands-on with tools like Data Factory, Databricks, Synapse, Logic Apps and more. It’s a chance to work on real, production-grade solutions while sharpening your cloud skills and growing alongside a strong, supportive team.

You’ll design and build scalable, automated data pipelines that bring together information from across the organisation. It’s a hands-on engineering role covering everything from data ingestion to transformation, modelling and delivery - all within a modern Azure environment.

Great data powers great decisions. We’re investing in our platform and need sharp, curious engineers to help us deliver faster, smarter insights., We’re one of the UK’s leading housing associations and developers. We were founded on a simple belief: high quality housing is vital for people’s health, happiness and security. Our purpose is simple - to provide social homes and landlord services that everyone can be proud of.

250,000 people call our properties ‘home’, and we’re proud to serve diverse communities across London, the South East and North West of England.

People are at the heart of our business and our success depends on employing the best people and getting the best from them. The foundation of everything that we are is built on our corporate values, which outline our core expectations and should be demonstrated at all times, and all levels, when representing L&Q - click here to read more.

At L&Q, we know that diversity and inclusion make us stronger - and they’re at the heart of everything we do.

When we recruit, we look at what really matters: your skills, experience, and potential. We’re proud to be recognised for creating an inclusive workplace. We’re a Disability Confident Leader (Level 3) and we’ve introduced our own Recruitment Advocate scheme to make sure every step of our hiring process is fair, transparent, and consistent. It’s all part of our commitment to ending discrimination and making L&Q a place where everyone feels welcome. Find out more here .

Sustainability is also at the heart of what we do. We recognise the responsibility we hold as one of the UK’s largest housing associations.