Data Engineer

r\n \r\n

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.\r\n
• Advanced Python knowledge for data processing and scripting.\r\n
• Proven experience with Spark and SQL for data engineering and analysis.\r\n
• Experience with data orchestration tools like Airflow.\r\n
• Hands-on experience with one or more cloud services (Azure, AWS, GCP).\r\n
• Proficiency in RDBMS/NoSQL data stores and appropriate use cases.\r\n
• Experience with Data as Code; version control, small and regular commits, unit tests, CI/CD, packaging, familiarity with containerization tools such as Docker (must have) and Kubernetes (plus).\r\n
• Solid understanding of the software development life cycle.\r\n
• Understanding of blockchain technologies and data structures.\r\n
• Knowledge of cryptography and its application in blockchain is a plus.\r\n
• Experience with blockchain indexing is a plus.\r\n
• Experience with Databricks for data ingestion and transformation is a plus.\r\n
• Familiarity with Delta Lake and data warehousing concepts is a plus.\r\n
• Strong communication, interpersonal and presentation skills.\r\n
• \r\n

Role and Responsibilities: \r\n \r\n

• Design, develop, and maintain scalable data pipelines using Python, PySpark and Databricks.\r\n
• Implement data orchestration workflows using Airflow or similar tools.\r\n
• Manage and optimize data storage solutions on Azure Cloud.\r\n
• Develop and maintain containerized microservices using Docker.\r\n
• Collaborate with cross-functional teams to ensure data quality and integrity.\r\n
• Support the setup of blockchains.\r\n
• Build tools to acquire data from blockchains and perform analysis on the data using Databricks.\r\n
• Support production infrastructure, including monitoring and automation using tools like Datadog.\r\n
• Participate in agile development practices, contributing to planning exercises and working in cross-functional groups.\r\n

\r\n