Lead Data Engineer
Role details
Job location
Tech stack
Job description
We are looking for a Lead Data Engineer as part of the technical authority group, and responsible for driving the end-to-end design, architecture, development, and deployment of data systems. You will lead and manage a small group of data engineers, ensuring productivity and progression in equal measure, as well as set best practices, and help deliver insights and data products that enable business decisions. A key part of the role is bringing in your expertise with and modern Azure tools and Snowflake data warehousing, to build robust, future-proof data platforms. As a technical expert, you have a true "the buck stops with me" attitude to resolving complex challenges, and know when to seek alternative pathways if delivery progress is being impeded. You will act as an advocate for technical excellence, and are continually up to date with industry trends and techniques with the ability to choose and decide which will be beneficial to apply in Tevalis software development.
This is a hybrid role with an expected 1 day per week at our offices on Bridgehead Business Park, Hessle.
Roles and Responsibilities
- Defining and building data warehouse and reporting solutions.
- Assist in improving optimisation, streamlining, automation, and security of existing data flows and storage solutions.
- Managing day to day operations of the data team.
- Chair daily standups and sprint ceremonies, distributing and overseeing the team's delivery of objectives and requests.
- Create high level documentation to be interpreted and understood by technical and non-technical audiences.
- Prioritise and effectively manage your time amongst several concurrent projects.
- Refine and automate regular processes, track issues, and document changes.
- Define and own the overall data architecture roadmap, ensuring it's aligned with business strategy, compliance, and scalability.
- Design, build, and maintain data pipelines, ETL/ELT workflows, data warehouses, data lakes, and real-time streaming where needed.
- Leverage existing Snowflake and Azure Fabric (and associated Azure data services, e.g. Data Factory) to build performant, observable, and secure data platforms.
- Lead and mentor a team of data engineers & analytics engineers in setting coding standards, architectural reviews, best practices, and peer code reviews.
- Ensure data governance, data quality, lineage, security and compliance are built into all data solutions.
- Understand business, data and reporting requirements by collaborating closely with product, engineering, operations, and non-technical teams to understand requirements, deliver data products, and embed data-driven decision-making.
- Optimise for cost, performance, operations across monitoring, alerting, scaling, and capacity planning.
- Stay up to date with newest features and capabilities in cloud technologies and drive adoption and best use.
- Provide technical leadership in estimation, technical debt, documentation, and trade-off decisions.
Requirements
Do you have experience in Technical support?, * Experience of troubleshooting problems both on-premises and in the cloud.
- Monitor database performance to avoid bottlenecks and critical conditions to ensure continued delivery of service.
- Excellent understanding of the SQL Server architecture.
- Excellent understanding of T-SQL queries, stored procedures, user-defined functions, views, indexes, and developer code associated with the database.
- Excellent analytical and problem-solving skills, identifying root causes and offering solutions.
- Solid knowledge of Snowflake, SQL Server, Azure SQL Database, and MongoDB.
- Building and improving Data Architecture on modern data platforms.
- Experience with data storage technologies such as data lakes, data warehouse, and traditional SQL databases.
- Data Warehouse and Data Modelling Design (primarily Snowflake).
- Data lake development using big data technologies or cloud native solutions.
- Complex Data integration pipelines with varied types and sources of data (direct integration, API based, ftp transfer).
- Deep level knowledge of how to create code with security, performance, and scalability in mind.
- An open-minded, can-do approach to software project delivery.
Benefits & conditions
Job Types: Full-time, Permanent
Pay: Up to £90,000.00 per year
Benefits:
- Additional leave
- Bereavement leave
- Company events
- Company pension
- Enhanced maternity leave
- Enhanced paternity leave
- Free parking
- Health & wellbeing programme
- Life insurance
- On-site gym
- On-site parking
- Referral programme
- Sick pay
