Senior Data Engineer
Spait Infotech Private Limited
yesterday
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
Senior Compensation
£ 95KJob location
Remote
Tech stack
Java
Agile Methodologies
Airflow
Amazon Web Services (AWS)
Azure
Google BigQuery
Computer Programming
Continuous Integration
Information Engineering
Data Governance
ETL
Data Security
Data Warehousing
Python
Meta-Data Management
SQL Databases
Data Ingestion
System Availability
Snowflake
Spark
GIT
Cloudformation
Containerization
Data Lake
Apache Flink
Terraform
Docker
Redshift
Databricks
Job description
- Design, develop, and maintain scalable, reliable, and secure data pipelines using modern data engineering technologies.
- Build data ingestion frameworks, ETL/ELT processes, and real-time streaming solutions.
- Architect, optimize, and monitor data warehouses, data lakes, and lakehouse environments.
- Collaborate with cross-functional teams to ensure data availability, quality, and performance.
- Implement best practices for data governance, security, lineage, and metadata management.
- Develop and maintain CI/CD pipelines for data engineering workloads.
- Troubleshoot complex data issues, ensuring high availability and resilience in production.
- Mentor junior engineers and contribute to architectural decisions and technical strategy.
Requirements
Do you have experience in Terraform?, * 6-10 years of hands-on experience as a Data Engineer or similar role.
- Strong programming experience in Python, Scala, or Java.
- Proven expertise with SQL and optimizing complex queries.
- Extensive experience with cloud platforms such as:
- AWS (preferred), Azure, or GCP
- Hands-on experience with:
- Data warehousing (e.g., Snowflake, Redshift, BigQuery)
- Data lake/lakehouse architectures
- Distributed processing frameworks (e.g., Spark, Databricks, Flink)
- Experience building ETL/ELT pipelines and working with workflow orchestration tools such as Airflow, dbt, or similar.
- Knowledge of CI/CD, Git, containerization (Docker), and infrastructure-as-code (Terraform or CloudFormation).
- Strong understanding of data modelling, data governance, security standards, and best practices.
- Experience in Agile development environments.
Benefits & conditions
Job Types: Full-time, Permanent
Pay: £75,000.00-£95,000.00 per year
Benefits:
- Work from home