Data Engineer with Iceberg

Platform teams at Acadian are responsible for building and supporting services that enable our development teams to leverage frameworks, tooling, and services across in-house and vendor PaaS/IaaS offerings. They define the architectural direction, best practices, and provide education on these services. This is a contributing role within the Platform Team responsible for a subset of these services, joining a cross-functional team of engineers to help define the architectural direction of the firm, while providing mentorship to junior team members. This role will be responsible for contributing to the design and technical oversight of various project deliveries. This role works hands on with the teams on various technology improvements and evaluations within our environment to enable our business and technology delivery groups. The candidate must have a strong architectural background and be extremely hands-on from a development perspective.

The Platform Data team specifically focuses on services that drive our practices around managing and visualizing data at Acadian. The team is responsible for Snowflake, Tableau, RDBMS systems like SQL Server and Postgres, automation for database deployments against these systems, and, in general, patterns and practices that advance our capability to process data.

What You ll Do:

• Design and mature DataOps services and tooling that support our cloud initiatives
• Development of scheduled jobs using Airflow
• Development of containers for use with ECS, Fargate, and EKS
• Development of data pipeline to feed data from various sources to Apache Kafka and load into Iceberg/snowflake.
• Consult on infrastructure projects to fit systems or infrastructure to the technical architecture and the needs of the end user community.

Mandatory Skills: Iceberg and the primary skills they are looking for. Python and Snowflake proficiency is needed as well., * Familiarity with Continuous Integration and Continuous Deployment techniques and tooling

• Knowledge of application stack architecture (e.g., microservices), PaaS development, and AWS
• Familiarity with scripting languages including but not limited to Bash
• Expertise in one of the following: python, Go, and/or C#
• Experience with Terraform or other IaC tools, (e.g., CloudFormation)
• Experience with Apache kafka and flink is preferred.
• Demonstrated expertise with Kubernetes
• Demonstrated expertise with Linux and Docker
• Strong interpersonal and communication skills
• Demonstrated analytical and problem-solving skills
• Ability to multi-task and work on multiple projects
• Strong experience with SQL Server, Postgres and snowflake
• Strong experience with ETL/ELT techniques