Data Engineer-Security Clearance Required

As an Data Engineer, you will play an important role in supporting live data services while contributing to platform improvements and automation initiatives. We are particularly interested in candidates who:

• Have experience with Pentaho, databases and SQL
• Are user-focused and comfortable engaging with stakeholders
• Enjoy problem-solving in fast-paced environments Show strong collaboration skills and eagerness to learn, * Pentaho Business Analytics (metadata & BI models)
• Pentaho Data Integration (PDI) and ETL workflow
• SQL query writing and troubleshooting
• Database fundamentals
• Git version control
• Bash scripting for Linux automation, deployment, and orchestration
• Python scripting for automation and tooling
• GitLab CI/CD pipeline configuration and orchestration
• Software development for building automation tools and workflows
• REST API integration for workflow automation

• Docker containerisation and orchestration AI/LLM tool usage (prompt engineering and developer assistance tools)

• Data virtualisation tools (e.g., Denodo)
• Monitoring and alerting platforms (Grafana, Dynatrace)
• Power BI or other data visualisation platforms
• Front-end testing tools (e.g., Cypress)
• Experience in frontline technical support environments

• has a deep appreciation of reproducible CI/CD pipelines which include automated tests, and knows how to deploy end-to-end to Production environments.
• has a deep working knowledge of their chosen programming language and related toolsets. Has applied software engineering best practices and design principles to data pipelines.
• understands the security implications related to developing and operating data processing platforms.
• has a breadth of experience across data tech and chooses the right tool for the right job. They understand data changes at scale, know how to ensure data quality at all stages and apply the fundamentals of data modelling based on usage.
• Is able to craft high-quality, resilient Data Pipelines for ingesting, transforming and exporting data. They are able to critically evaluate and choose appropriate orchestration/choreography tools, know how to exploit parallelism and pushdown/offload, build in monitoring, are aware of the complexity of testing data pipelines, and follow functional data engineering principles.
• Is able to craft views and queries that answer analytical questions in a concise and efficient manner using SQL. Has a deep experience of at least one, preferably more, SQL dialects. Is able to analyse and optimise queries and understand their dependence on indexing and partitioning. Is able to effectively decompose complex data projections using CTEs.
• Has an appreciation of key data governance concerns and experience in designing and implementing solutions for them.
• Primarily is willing to help others. Is capable and happy to pair with others. Seeks peer review on work. Considers processes, clients needs as well as technology when working. Also has good interpersonal skills
• Understands the practices and principles underpinning agile software development. Able to adjust their ways of working based on the client’s maturity, * Have Permanent Right to Work in The UK
• Be happy to have evidence of Right to work and Enhanced DBS checks completed
• Be eligible for SE Clearance/already have Security Clearance
• Must have lived continuously in the UK for the last 5 years (no absence of more than 6 months)

Counter, a consultancy brand under the Northcoders group, has a model where we help clients attract, onboard, upskill and integrate graduates into their businesses, in order to give them a seamless start to long and successful careers there. Counter teams are formed for our clients by deploying a Counter Tech Lead to work within the client’s organisation in order to hire appropriate team members from graduate and professional communities and then to work in collaboration with them directly on the clients projects. Providing support, extra training and all the guidance to fast-track the individuals’ journey within that organisation. For this engagement, Counter has been contracted to provide a team and will support a Government Department., Counter, a consultancy brand under the Northcoders Group, adopts a model where we help clients attract, onboard, upskill and integrate talent into their businesses, in order to give the team a seamless start to long and successful careers there. Counter teams are formed for our clients by deploying a Counter Tech Lead to work within the client’s organisation in order to hire appropriate team members and then to work in collaboration with them directly on live clients projects. Providing support, extra training and all the guidance to fast-track the individuals’ journey within that organisation.