> Markdown version of [/jobs/ext/25917-data-engineer](https://www.wearedevelopers.com/jobs/ext/25917-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Brook Green Supply Ltd - **Location:** London, UK (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Agile Methodology, Airflow, Big Data, Databases, Continuous Integration, Data Validation, Information Engineering, Data Integrity, Extract Transform Load (ETL), Data Warehousing, Relational Databases, Software Debugging, Distributed Systems, Monitoring of Systems, Python (Programming Language), PostgreSQL, Meta-Data Management, MySQL, NumPy, Performance Tuning, Query Optimization, Standard Sql, Data Streaming, Tableau (Software), Data Processing, Apache Spark, Git, Pandas, Containerization, Kubernetes, Information Technology, Apache Kafka, Graphql, Vertica, Stream Analytics, Software Version Control, Data Pipelines, Docker - **Published:** May 24, 2026 - **Apply:** https://www.careerjet.co.uk/jobad/gb36518ea1da07fb5e8d64eb6152ca9374 ## About the Role * Great SQL skills with experience in complex query optimization * Strong Python programming skills with experience in data processing libraries (pandas, NumPy, Apache Spark) * Hands-on experience building and maintaining data ingestion pipelines * Proven track record of optimising queries, code, and system performance * Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow) * Knowledge of distributed computing concepts and big data technologies * Experience with version control systems (Git) and CI/CD practices * Experience with relational databases (PostgreSQL, MySQL or similar) * Experience with containerization technologies (Docker, Kubernetes) * Experience with data orchestration tools (Apache Airflow or Dagster) * Understanding of data warehousing concepts and dimensional modelling * Understanding of cloud platforms using infrastructure-as-code (IaC) approaches * Knowledge of streaming data processing and real-time analytics * Experience with data quality and monitoring tools Analytical and Soft Skills: * Strong analytical mindset with attention to detail * Excellent problem-solving and debugging skills * Ability to work independently and manage priorities * Strong communication skills and ability to explain technical concepts to non-technical stakeholders * Experience working in agile development environments * Passion for continuous learning and staying current with emerging technologies Preferred Qualifications: * Bachelor's degree in Computer Science, Engineering, Mathematics, or related field * 2-5 years of experience in data engineering or related roles * Experience working in the Energy industry Data Engineer * Experience with columnar databases (ClickHouse or similar) * Experience with open-source data catalogue and metadata management tools * Experience with performance tuning * Experience with data visualisation tools (Tableau) * Experience with GraphQL, Even if you don't tick every box, we would still love to hear from you. If you are detail-oriented, analytical, and motivated to grow your career in the energy industry, we will give you the training and support to succeed. Brook Green Supply is committed to ensuring equal opportunities, fairness, dignity, and respect for all employees and applicants. Diversity makes us stronger, and we welcome applicants from all backgrounds. ## Description * Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools * Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements * Develop data models and schemas that support analytics, business intelligence and product needs * Monitor, troubleshoot, and optimise data pipeline performance and reliability * Collaborate with stakeholders, analysts and product team to understand data requirements * Implement data quality checks and validation processes to ensure data integrity * Participate in architecture decisions and contribute to technical roadmap planning Required Skills and ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.](https://www.wearedevelopers.com/videos/562-vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Branch your database like your code: How schema changes and pull requests go hand in hand](https://www.wearedevelopers.com/videos/350-branch-your-database-like-your-code-how-schema-changes-and-pull-requests-go-hand-in-hand) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [A Guide to Green Tech and Green IT Careers](https://www.wearedevelopers.com/magazine/374-a-guide-to-green-tech-and-green-it-careers) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Best Companies to work for in London: Top 25 Companies in 2023](https://www.wearedevelopers.com/magazine/187-best-companies-to-work-for-in-london-top-25-companies-in-2023)