Data Engineer

As a Data Engineer, you’ll work across the data lifecycle, with a strong focus on data ingestion, transformation, migration, warehousing and data management.You’ll be working with AWS data technologies alongside IBM DataStage, helping to build and maintain robust data integration and transformation processes across large-scale data environments., * Design, develop and maintain enterprise-scale data ingestion and transformation processes.

• Develop and support ETL and data migration pipelines across both batch and real-time processing.
• Support the development and management of large-scale enterprise data warehouse solutions.
• Build and maintain data integration and transformation processes using IBM DataStage.
• Work with AWS technologies including Amazon Redshift, Amazon S3, AWS Glue and Amazon QuickSight.
• Develop and optimise SQL-based data transformations and work with relational databases.
• Support database storage, management and wider data-platform activities.
• Apply DevSecOps principles to data engineering and deployment processes.
• Use GitHub/GitLab for source control and collaborative engineering.
• Contribute to Java development where required within the wider data platform.

You’ll need strong commercial data engineering experience, particularly across enterprise-scale ingestion, transformation, warehousing and database environments.

Essential skills and experience

• IBM DataStage
• Amazon Redshift
• Amazon S3
• AWS Glue
• Amazon QuickSight
• AWS data/cloud environments
• ETL and data migration
• Batch and real-time data processing
• Enterprise data warehouse development
• Large-scale data ingestion and transformation
• SQL
• Relational databases
• Database / DBA knowledge
• DevSecOps
• Java
• GitHub / GitLab
• Minimum 5 years’ continuous/unbroken UK residency