Cloud Data Engineer
Role details
Job location
Tech stack
Job description
As a Senior Cloud Data Engineer, you'll design and implement functionalities, focusing on Data Engineering tasks. You'll be working with semi-structured and non-structured data to ingest and distribute in cloud environments to modernize data products and distribution channels. You'll drive the software development lifecycle for continuous data delivery and lead the evaluation and adoption of emerging technologies., * Create and maintain optimal data pipeline architecture,
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Work with stakeholders, including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Keep our data separated and secure across national boundaries through multiple data centers and Azure regions.
- Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Work with data and analytics experts to strive for greater functionality in our data systems.
Requirements
- Depending on seniority relevant experience in Data Platforms (in Financial Services industry), Azure's PaaS/SaaS offerings (Fabric, Synapse, Purview, ADF, Azure Data Lake Storage etc.)
- Demonstrable experience in a similar role, with a focus on cloud distributed data processing platform for spark.
- Solid experience with Azure: Synapse Analytics, Data Factory, Data Lake, Databricks, Microsoft Purview, Monitor, SQL Database, SQL Managed Instance, Stream Analytics, Cosmos DB, Storage Services, Azure Functions, Log Analytics, Serverless Architecture
- Strong proficiency in Spark, SQL, and Python/scala/Java.
- Knowledge and understanding of Snowflake
- Knowledge of security best practices (e.g., using Azure Key Vault, IAM, RBAC, Monitor etc.).
- Proficient in integrating, transforming, and consolidating data from various structured and unstructured data systems into a structure that is suitable for building analytics solutions.
- Understand the data through exploration, experience with processes related to data retention, validation, visualization, preparation, matching, fragmentation, segmentation, and enhancement.
- Demonstrates ability to understand business requirements and the implications of those requirements on current and future roadmaps.
Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.