> Markdown version of [/jobs/ext/1909283-data-engineer](https://www.wearedevelopers.com/jobs/ext/1909283-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** NICHE SKILLS LIMITED - **Location:** Watford, UK - **Experience:** Expert - **Salary:** £39,000.0 - £40,000.0 - **Contract:** Permanent contract - **Skills:** Microsoft Excel, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Amazon Web Services, Business Analytics Applications, Data Analysis, Software Applications, Microsoft Azure, Bash Shell, Business Software, Cloud Computing, Cloud Engineering, Computer Programming, Information Technology Consulting, Databases, Data Architecture, Data Validation, Information Engineering, Data Infrastructure, Data Integration, Extract Transform Load (ETL), Data Warehousing, DevOps, Dimensional Modeling, Document-Oriented Databases, Github, Python (Programming Language), PostgreSQL, Machine Learning, Microsoft Data Access Components, Microsoft SQL Server, SQL Azure, MySQL, Oracle Databases, Windows PowerShell, Scrum Methodology, Recommender Systems, Power BI, Cloud Services, Standard Sql, Azure Machine Learning, Search Technologies, Systems Integration, Tableau (Software), Data Processing, Google Cloud, Cloud Platform System, Chatbots, Data Ingestion, Microsoft Power Automate, Azure Data Factory, Retrieval-Augmented Generation, Large Language Models, Prompt Engineering, Apache Spark, Generative AI, Git, Powerquery, Microsoft Fabric, Data Lakes, AI Platforms, Pyspark, Deployment Automation, Data Analytics, Data Management, Cloud Migration, Azure Synapse Analytics, Data Pipelines, Databricks - **Published:** August 4, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=ab6a816247f39133 ## About the Role * At least five years of professional experience in data engineering, data analytics, Artificial Intelligence, cloud technology, or IT consulting. * Proven experience delivering end-to-end data and analytics solutions. * Strong understanding of enterprise data architecture and modern cloud-based data platforms. * Experience gathering requirements and working directly with business stakeholders. * Strong analytical, communication, presentation, and problem-solving skills. * Ability to translate business requirements into practical technical solutions. * Experience producing technical documentation and solution designs. Technical Skills Candidates should have strong experience in several of the following areas: Programming and Data Processing * Python * SQL * PySpark * PowerShell or Bash Microsoft Data Platform * Microsoft Fabric * Azure Data Factory * Fabric Data Factory * Azure Synapse Analytics * Azure Databricks * Apache Spark * OneLake * Data pipelines and data integration Databases and Data Architecture * Microsoft SQL Server * Azure SQL Database * PostgreSQL * MySQL * Oracle Database * Data warehouse design * Data lake and lakehouse architecture * Dimensional modelling * Star and snowflake schemas Analytics and Reporting * Power BI * DAX * Power Query * Advanced Microsoft Excel * Tableau experience is desirable Artificial Intelligence * Machine learning * Predictive analytics * Azure AI Services * Azure OpenAI Service * Microsoft Copilot Studio * Prompt engineering * Large Language Models * Retrieval-Augmented Generation * Vector databases * AI agents * Intelligent automation * Document intelligence DevOps and Delivery * Git * GitHub * Azure DevOps * CI/CD pipelines * Infrastructure and deployment automation Experience with AWS or Google Cloud Platform would also be beneficial. What We Are Looking For We are looking for someone who: * Combines strong technical ability with a consulting mindset. * Is confident communicating with clients and senior stakeholders. * Can independently investigate complex business and technical problems. * Understands how to balance innovation with security, governance, and maintainability. * Can manage multiple priorities and deliver high-quality work. * Is collaborative, self-motivated, and results-oriented. * Is committed to continuous learning and professional development. ## Description We help organisations improve how they collect, manage, analyse, and use data. Our consultants work closely with clients to design scalable data platforms, build automated workflows, create business intelligence solutions, and implement AI-powered applications that support better decision-making and digital transformation., You will work across the complete data lifecycle, from data ingestion and engineering through to analytics, machine learning, Generative AI, and intelligent automation. The role combines hands-on technical delivery with consultancy and stakeholder engagement. You will work with clients to understand their business challenges, translate requirements into practical technology solutions, and deliver secure, scalable, and governed data and AI platforms. You may contribute to projects involving Microsoft Fabric, Azure Data Factory, Databricks, Power BI, Azure OpenAI, machine learning, Retrieval-Augmented Generation, and AI-powered business automation. Key ResponsibilitiesConsulting and Solution Design * Engage with clients and business stakeholders to understand their objectives, processes, and data requirements. * Translate business needs into scalable data, analytics, and AI solutions. * Produce solution designs, architecture diagrams, technical documentation, and implementation roadmaps. * Recommend best practices for data architecture, governance, security, cloud adoption, and responsible AI. * Identify opportunities to improve operations through automation, analytics, and Artificial Intelligence. * Provide technical leadership throughout project delivery. Data Engineering * Design, develop, and maintain scalable ETL and ELT data pipelines. * Integrate data from databases, APIs, files, cloud applications, and external systems. * Build and manage data warehouses, data lakes, lakehouses, and cloud-based data platforms. * Develop data models for reporting, analytics, machine learning, and AI workloads. * Implement data validation, quality, governance, security, and monitoring controls. * Troubleshoot and optimise data pipelines, databases, and processing workloads. * Automate data ingestion and transformation processes. Data Analytics and Business Intelligence * Analyse complex datasets to identify business trends, risks, opportunities, and performance issues. * Develop interactive Power BI dashboards, executive reports, KPIs, and analytical models. * Work with stakeholders to define reporting requirements and business metrics. * Perform descriptive, diagnostic, predictive, and prescriptive analytics. * Present findings clearly to technical and non-technical audiences. * Deliver actionable insights that support strategic and operational decision-making. Artificial Intelligence and Automation * Design and implement machine-learning and AI solutions for real business challenges. * Develop predictive models, forecasting solutions, recommendation engines, and intelligent workflows. * Build Generative AI solutions using Large Language Models. * Implement AI assistants, chatbots, document intelligence, semantic search, and Retrieval-Augmented Generation solutions. * Integrate Azure OpenAI and other AI services into business applications and data platforms. * Evaluate model performance and improve accuracy, reliability, and usability. * Apply responsible AI principles, including security, transparency, privacy, human oversight, and ethical use. Collaboration and Delivery * Work closely with developers, architects, project managers, business analysts, and client teams. * Participate in Agile and Scrum ceremonies. * Maintain clear technical and operational documentation. * Support testing, deployment, production readiness, and knowledge transfer. * Mentor junior consultants and share technical knowledge. * Stay current with developments in cloud data platforms, analytics, machine learning, and Generative AI., The successful candidate may contribute to the delivery of an Intelligent Customer Operations Data Platform. The project could involve: * Integrating customer, sales, service, operational, and document data. * Building a Microsoft Fabric lakehouse using a medallion data architecture. * Developing automated data pipelines and data-quality controls. * Creating executive and operational Power BI dashboards. * Building predictive models for customer churn, service demand, and operational risk. * Developing an Azure OpenAI-powered assistant using Retrieval-Augmented Generation. * Implementing security, governance, monitoring, and responsible AI controls. * Deploying solutions through automated CI/CD pipelines. Key Deliverables * Secure and scalable data pipelines. * Modern data warehouse, lakehouse, and cloud data solutions. * Governed and reusable enterprise data models. * Interactive Power BI reports and dashboards. * Predictive machine-learning models. * Generative AI and intelligent automation solutions. * Technical documentation and architecture designs. * Actionable insights that support business decision-making. Why Join Niche Skills? * Work on modern data, analytics, cloud, and Artificial Intelligence projects. * Use technologies including Microsoft Fabric, Azure, Power BI, Azure OpenAI, and Databricks. * Work directly with clients and influence solution design. * Gain exposure to projects across the complete data and AI lifecycle. * Join a collaborative and growing technology consultancy. * Access opportunities for professional development and technical leadership. * Benefit from flexible hybrid, remote, and client-site working arrangements. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Branch your database like your code: How schema changes and pull requests go hand in hand](https://www.wearedevelopers.com/videos/350-branch-your-database-like-your-code-how-schema-changes-and-pull-requests-go-hand-in-hand) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Where To Find Software Engineering Jobs](https://www.wearedevelopers.com/magazine/396-where-to-find-software-engineering-jobs)