Senior Data Engineer
Role details
Job location
Tech stack
Job description
As an integral part of our team, you will drive the day to day operation of Moonfare, which in turn will influence how our product evolves and how customers interact with our product. You will be joining a small team of Data Engineers, ensuring that real-time data is available to our analysts through the design and development of robust data pipelines. As our business is rapidly expanding, so are our data sources, so you will be responsible for initiating scalable and streamlined solutions to provide reliable data to the business in a secure and timely manner.
You will:
- Manage and scale Moonfares's BI data infrastructure by developing and maintaining our data pipelines from ingestion through to storage
- Model data using solid data lake/warehouse concepts and optimize performance(partitioning, file formats, performance tuning)
- As a central figure of a small team, you will champion standards, processes and best practices in your area
- implement least-privilege IAM, encryption, data classification, PII handling, and auditability and contribute to data governance policies
- Conduct data analysis experiments, providing bespoke solutions to our stakeholders giving them the information they need to succeed
- collaborate with analysts and domain stakeholders to translate requirements into robust data solutions, review code, and mentor peers
- High quality data is at the core of Moonfare's success, you will help us continue our standards through data cleansing, coding consistency and data availability
- We want to hear your ideas! You will bring all of your previous experiences and thoughts to the table and help us bring in new tools, topics and methodologies to the BI Team
Requirements
Do you have experience in Tableau?, * 5+ years working experience as a Data Engineer, ideally having worked within a regulated financial services environment
- You are an expert in designing and developing robust Data Pipelines using technologies such as Apache Spark,Hadoop ecosystem with a deep understanding of data processing
- You have a strong background in cloud infrastructure (especially AWS) and hands-on experience of data lakes and modern data warehouses, including familiarity with orchestration, security, and network
- You have proven programming experience with Python and Scala and you are also happy to review code
- You work with relational databases, data warehouses and advanced SQL queries
- You have experience with CI/CD & bash scripting and deployment/release best practices
- You have practical experience with Docker and working knowledge of Kubernetes
- You have working knowledge of DBT (models, tests, documentation) and a general understanding of Tableau or similar BI tools
- You collaborate with analysts and other stakeholders to understand data requirements and provide efficient data solutions
- You contribute to the development and maintenance of data governance and security practices
- Excellent written and verbal communication skills in English and have a clear and adaptive communication style
- You are solution-oriented and have excellent attention to detail
- You have creative problem-solving abilities and a passion for data