Data Engineer

We are seeking a highly skilled Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and architectures that support our organisation's data analytics and business intelligence initiatives. This role offers an exciting opportunity to work with cutting-edge technologies and contribute to the optimisation of data processes across various platforms. The successful applicant will possess strong technical expertise, particularly in cloud-based solutions, big data frameworks, and database management systems., * Develop, construct, test, and maintain robust data pipelines using tools such as Apache Spark, Hadoop, and Informatica.

• Design and implement efficient data models and architecture for data warehouses and lakes using Oracle, Microsoft SQL Server, and other relational databases.
• Manage and optimise large-scale datasets within cloud environments, primarily AWS, ensuring high performance and scalability.
• Write complex SQL queries to extract insights from diverse data sources including Oracle, SQL Server, and big data platforms.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.
• Automate workflows through shell scripting (Bash/Unix shell) and VBA to streamline data processing tasks.
• Perform database design activities to ensure data integrity, security, and compliance with organisational standards.
• Monitor system performance; troubleshoot issues related to ETL processes or data pipelines; implement improvements as needed.
• Document architecture designs, processes, and workflows for future reference and knowledge sharing.

• Proven experience in Data Engineering roles with a strong background in big data technologies such as Hadoop, Apache Hive, Spark, and related frameworks.
• Hands-on expertise in programming languages including Python and Java for developing scalable data solutions.
• Extensive knowledge of cloud platforms, particularly AWS services relevant to data storage and processing.
• Solid understanding of relational databases like Oracle and Microsoft SQL Server along with database design principles.
• Experience working with ETL tools such as Informatica is highly desirable.
• Proficiency in SQL optimisation techniques; analysis skills for interpreting complex datasets.
• Familiarity with shell scripting (Bash/Unix) and VBA for automation purposes.
• Strong analytical skills combined with the ability to communicate technical concepts effectively to non-technical stakeholders. This role offers an excellent opportunity for a motivated Data Engineer eager to develop innovative solutions within a collaborative environment. Applicants should demonstrate a passion for technology-driven problem solving coupled with a comprehensive understanding of modern data architectures.