Senior Data Engineer

Kainos

Belfast, UK

13 days ago

Apply on www.collegerecruiter.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Working hours

Regular working hours

Job source

www.collegerecruiter.com

Tech stack

Java (Programming Language) Artificial Intelligence Amazon Web Services Automation of Tests Microsoft Azure Big Data Software Quality Continuous Delivery Continuous Integration Extract Transform Load (ETL) Persistent Data Structure Data Stores

+15 more

Data Systems Data Visualization Software Design Patterns Distributed Computing Environment Python (Programming Language) Performance Tuning Standard Sql Software Engineering SQL Databases Data Processing Google Cloud Change Data Capture Apache Kafka Data Pipelines Databricks

Job description

As a Senior Data Engineer (Senior Associate) at Kainos, you will be responsible for designing and developing data processing and data persistence software components for solutions which handle data at scale. Working in agile teams, Senior Data Engineers provide strong development leadership and take responsibility for significant technical components of data systems. You will work within a multi-skilled agile team to design and develop large-scale data processing software to meet user needs in demanding production environments., * Working to develop data processing software primarily for deployment in Big Data technologies. The role encompasses the full software lifecycle including design, code, test and defect resolution.

Working with Architects and Lead Engineers to ensure the software supports non-functional needs.
Collaborating with colleagues to resolve implementation challenges and ensure code quality and maintainability remains high. Leads by example in code quality.
Working with operations teams to ensure operational readiness.
Advising customers and managers on the estimated effort and technical implications of user stories and user journeys.
Coaching and mentoring team members.

Requirements

Strong software development experience in one of Java, Scala, or Python
Software development experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
Experience of developing substantial components for large-scale data processing solutions and deploying into a production environment
Proficient in SQL and SQL extensions for analytical queries
Solid understanding of ETL/ELT data processing pipelines and design patterns
Aware of key features and pitfalls of distributed data processing frameworks, data stores and data serialisation formats
Able to write quality, testable code and has experience of automated testing
Experience with Continuous Integration and Continuous Deployment techniques, * Experience of performance tuning
Experience of data visualisation and complex data transformations
Experience with steaming and event-processing architectures including technologies such as Kafka and change-data-capture (CDC) products
Expertise in continuous improvement and sharing input on data best practice
Practical experience with AI technologies, tools, processes and delivery