Data Engineer

Come and support Company.info in the biggest transformation in the company's history by strengthening our data engineering capabilities. We are seeking a highly skilled and motivated Senior Data Engineer to join our team. You will be responsible for designing, building, and maintaining our data pipelines and work closely with other teams to ensure that our data is clean, accurate, and readily available to support our client's business decisions., You will work on ingesting, processing, and integrating data products for a specific domain. Some of the tasks that you will be involved with include:

• Design and implement data modelling architecture.
• Build production grade data pipelines and different processing paradigms (streaming, batch, and graph).
• Ingest data from a wide variety of sources with different sizes and formats.
• Use automated deployment tools and templates to deploy the solutions! (CI/CD)
• Work with Spark Structured Streaming, Delta Live Tables, and Change Data Capture to refresh data products that require low latency!

Our stack is mainly composed of the following tools: Python, Databricks, PySpark, AWS S3, EventBridge, and Lambdas.

Culture / the team …

You will be working together in a passionate team that continues to strive for being the first/best in the market, modern management, no hierarchy, and an informal atmosphere. Our Product & Tech department consists of 40 people. We are all passionate about our products, technology, and data.

• You have at least 5+ years of relevant working experience in the IT or data sector.
• Extensive experience in production grade data ingestion, processing, and modelling.
• Experience with cloud environments (preferably AWS).
• Preferably you have a Bachelor or Master in Computer Science, Software engineering, AI or a related field.

Key technical skills and requirements

• Proficiency with Spark (preferably Databricks platform).
• Extensive experience with software development (Python).
• Experience with shipping production grade software with CI/CD.
• Experience with AWS services (Lambda, Eventbridge), data stores (MySQL, PostgreSQL, Elasticsearch), and infrastructure as code (Terraform, CloudFormation).

Competencies

• Strong sense of ownership and great problem-solving skills.
• Drive to learn our data domains and develop innovative solutions.
• Like to coach more junior members and achieve team goals.
• Great communication skills (written and spoken), in English.

Currently, the data engineering chapter consists of five data engineers, who work together in one of the domain teams. We expect the data engineering chapter to grow to 6-8 people in the next 2 years. In terms of management style, the organization is flat, relaxed, and clear. The atmosphere is open, pleasant, social, and fun. In this role, you will have the opportunity to specialize in specific knowledge areas and become a technical lead for this domain. So lots of opportunities for personal and technical development and growth!

This is what you'll get from us…

We offer you a dynamic environment, international oriented with innovative, challenging, and diverse projects, using cutting-edge technology and directly benefiting our clients and our business managers at the same time.

• € 70.000 - € 78.000 euro per year including 8% holiday allowance, based on 40 hours per week.
• Profit sharing.
• Great premium free pension scheme.
• Flexible working hours and working in a hybrid way.
• We will make sure you have all the software and hardware (your MacBook of choice) required to perform at your highest level.
• Because we work with the latest technologies, we offer the opportunity to keep on developing yourself by taking courses, attending (and presenting at) conferences, e.g. Free full access to Databricks Academy.
• Very modern working environment, the office is new and specially furnished to the wishes of Company.info. The office is centrally located in Amsterdam and easily accessible by public transport.

Functie-eisen