DataOps Engineer
Role details
Job location
Tech stack
Job description
The DataOps Engineer is responsible for the development, implementation, and maintenance of a data platform serving the company's data lake: data acquisition, transformation pipelines, presentation, data access and supporting infrastructure. This role focuses on optimizing data flow and collection for cross-functional teams. The ideal candidate has a deep understanding of data engineering, its principles and best practices, DevOps and software development principles, and is committed to enhancing data management processes., 1. Pipeline Development & Management:
- Design, build, and maintain scalable and reliable data pipelines.
- Automate data ingestion, transformation, and delivery processes.
- Ensure data quality, integrity, and reliability throughout the data lifecycle.
Infrastructure Management:
- Manage cloud data infrastructure.
- Implement and optimize data storage solutions.
- Monitor and ensure the performance, scalability, and security of data platforms.
- Setting up alerts to detect anomalies
Collaboration & Support:
- Collaborate with data scientists, analysts, Engineers and other stakeholders to understand data requirements and deliver optimal solutions.
- Provide support for data-related issues and troubleshooting.
- Assist in the development and implementation of data governance and security policies including access management.
Continuous Improvement:
- Implement CI/CD processes for data workflows.
- Optimize existing processes and frameworks for better performance and cost-efficiency.
- Stay updated with the latest industry trends and technologies to continuously improve data operations.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Master's degree is a plus.
- 3+ years of experience in DataOps, Data Engineering, or a related role.
- Proven experience with data pipeline tools (Airflow).
- Hands-on experience with cloud platforms (AWS, Azure, Google Cloud) and data services (Snowflake).
- Proficiency in SQL and experience with relational databases.
- Strong programming skills in Python.
- Experience with IaC and CI/CD tools (Terraform, Jenkins, GitHub, Concourse CI).
- Knowledge of data warehousing and ETL/ELT processes.
Soft Skills:
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Ability to work independently and as part of a team.
- Attention to detail and a commitment to data quality.
Benefits & conditions
Working with a pioneering provider of eCommerce solutions you will have the opportunity to join an international company who are growing massively, we encourage ambition and creativity.
Plus, you will get:
- 25 Days Annual leave, increasing to 26 days after 12 months in the business
- Enhanced Company Pension (Matched up to 5% & Salary Sacrifice)
- Healthcare Cashplan with Medicash
- Private Healthcare with Aviva
- Life Insurance with AIG
- Happl, our benefit platform which provides access to pre-negotiated discounts on a wide variety of services including entertainment, food, and fitness.
- Stock / Equity
Check out the links below to see what our CEO Justin Floyd has to say about our plans for growth for the year ahead, and to see our latest video on the mission we're on!
RedCloud I We're growing!
RedCloud I Red101 App I Open Commerce
