Data engineer in high-volume systems

Описание:

At BETWEEN, we offer you the opportunities you need to achieve your professional goals. If you want to develop your skills in IT and Engineering, this is your next destination! We provide support across a wide range of areas, including Firmware, Software, BI, Design, Project Management, and many more. We also develop bespoke technology solutions through our BETWEEN Labs. We are looking for a Data Engineer to join a technology product team that works daily with large volumes of information, developing platforms and pipelines that turn data into business decisions. If you enjoy designing ETL processes, working with Python and SQL, and are motivated to develop scalable cloud solutions, this could be your next challenge.

Задачи:

• Design and develop data pipelines using Python.
• Build, optimize, and maintain ETL processes.
• Work with analytical databases and large volumes of information.
• Participate in workflow orchestration using Airflow.
• Develop solutions deployed in cloud environments.
• Contribute to CI/CD and version control practices using Git.
• Actively participate in technical decisions within the team.
• Work in product-oriented Agile environments.

Требования:

More than 4 years of experience working with Python and SQL in data environments. Experience designing, developing, and maintaining ETL processes. Solid knowledge of SQL (joins, subqueries, aggregate functions, CTEs, indexes, primary and foreign keys, database normalization concepts). AWS (S3, Redshift, Lambda, Secrets Manager, SSO, etc.) or equivalent services in Google Cloud or Azure. Columnar databases (Redshift). Experience working with code repositories and collaborative development methodologies. Будет плюсом: Git; CI/CD; Apache Airflow; Docker and Kubernetes; Databricks

Условия:

Hybrid model with 1 day a week in the office. You will participate in high-impact technology projects involving large volumes of data. You will work with leading cloud technologies. You will be part of a collaborative environment where you can continue to grow professionally.

• Health insurance with Alan.
• Training opportunities: languages, certifications, boot camps, and much more.
• Lots of events: BETWEEN Night, paddle tennis tournament, and much more.
• Flexible compensation (restaurant, transport, childcare).
• Incentives for validating or recommending talent.

More than 4 years of experience working with Python and SQL in data environments. Experience designing, developing, and maintaining ETL processes. Solid knowledge of SQL (joins, subqueries, aggregate functions, CTEs, indexes, primary and foreign keys, database normalization concepts). AWS (S3, Redshift, Lambda, Secrets Manager, SSO, etc.) or equivalent services in Google Cloud or Azure. Columnar databases (Redshift). Experience working with code repositories and collaborative development methodologies. Будет плюсом: Git; CI/CD; Apache Airflow; Docker and Kubernetes; Databricks