Data Engineer - Edinburgh

• Contribute to the development and delivery of solutions to integrate new datasets into the data warehouse
• Support the development of new data products in collaboration with the Senior Data & AI Product Manager and other stakeholders
• Assist in migrating the data warehouse from on-prem to AWS cloud
• Ensure technical resiliency of all data integration solutions and services
• Enhance and support existing data product outputs for both internal and external customers
• Collaborate with technical colleagues across the organization to design robust data integration solutions
• Demonstrate excellent, sustainable, and collaborative software development practices
• Actively participate in all team events, leading when specialist knowledge is required and supporting the team to improve their processes
• Troubleshoot and fix development and production problems across multiple environments and operating platforms
• Engage with wider communities of practice to share knowledge, techniques, and experiences
• Ensure high quality of developed solutions through unit tests and code quality tools
• Comply with non-functional software requirements such as accessibility, security, performance, and maintainability
• Support and deliver disaster recovery assurance of digital services.

Technologies:

• AI
• AWS
• CI/CD
• Cloud
• Data Warehouse
• DevOps
• Docker
• GitLab
• Grafana
• Support
• Kanban
• OpenShift
• PostgreSQL
• Product Manager
• Python
• REST
• SQL
• Security
• TDD
• UX UI Design

More:

This role is a 6-month contract position, offered outside IR35, with a day rate of £470. We're focused on collaboratively developing internal casework-profiling solutions using data extraction and processing techniques applied to semi-structured and unstructured documents. Our goal is to enhance our data analytics capabilities by creating a Data Warehouse that supports robust data governance and management.

• Significant commercial experience with the following technologies:
• • Python
• • PostgreSQL
• • REST APIs
• • Modern DevOps and CI/CD practices and tooling, including Docker, GitLab CI, AWS CodePipeline, AWS CDK, and AWS CloudFormation
• Significant and demonstrable commercial experience in:
• • SQL, data transformation, and analysis
• • Delivering high-quality software collaboratively in high-performing, cross-functional development teams
• • Implementing data ETLs, data streaming systems, and data integration solutions
• • Working in Agile delivery models, such as Scrum and/or Kanban frameworks
• Desirable qualifications include:
• • Data warehousing
• • Hybrid on-premises/cloud solutions
• • AWS Glue, Step Functions, Lambda functions, S3, RDS, and Data Migration Service
• • Using testing tools for unit testing, including system test automation frameworks
• • Openshift
• • PostGIS for PostgreSQL
• • Designing and implementing solutions using service and event-based architectures
• • Monitoring, alerting, and intelligence tools and processes, including Grafana
• • Human-centered, research-driven, inclusive design practices
• • Developing within Digital First or GDS quality standards
• • TDD