Senior Data Engineer

Varalakshmi Solutions Limited
North Mymms, United Kingdom
3 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
£ 60K

Job location

Remote
North Mymms, United Kingdom

Tech stack

Java
Agile Methodologies
Amazon Web Services (AWS)
Azure
Cloud Computing
Data Governance
ETL
Data Security
Database Design
DevOps
Hadoop
Python
Machine Learning
NoSQL
Cloud Services
SQL Databases
Data Streaming
Systems Integration
Consumer Database Management System
Google Cloud Platform
Spark
Data Lake
Information Technology
Kafka
Data Management
Programming Languages

Job description

Pipeline Development: Build and maintain scalable, robust ETL/ELT pipelines to ingest, clean, and transform data from diverse sources.

Infrastructure Management: Design, create, and maintain data warehouses, data lakes, and database architectures.

Data Optimization: Enhance data flow, collection, and storage to ensure data quality and reliability.

System Integration: Integrate data products into existing systems and business processes.

Collaboration: Work with data scientists and analysts to support machine learning, modelling, and reporting initiatives.

Security & Compliance: Implement data security, privacy policies, and best practices for data governance.

Requirements

Do you have experience in Spark?, Do you have a Bachelor's degree?, Technical Proficiency: Strong expertise in SQL and programming languages, specifically Python, Scala, or Java.

Data Technologies: Experience with big data tools and frameworks (e.g., Spark, Kafka, Hadoop).

Cloud Platforms: Proficiency in cloud services (e.g., AWS, Azure, GCP).

Data Modeling: Understanding of database design, normalization, and data modeling patterns.

Database Management: Experience with relational and NoSQL databases.

Education: Typically requires a bachelor's degree in Computer Science, Engineering, or a related field.

Core Competencies

Problem-Solving: Strong analytical skills to diagnose data quality issues and optimize system performance.

Communication: Ability to explain technical concepts to non-technical stakeholders.

Agile Development: Experience with DevOps practices and iterative, fast-paced development.

Apply for this position