Senior Data Engineer

Anywhere, Inc
Madison, United States of America
12 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Madison, United States of America

Tech stack

Java
API
Artificial Intelligence
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Code Review
Continuous Delivery
Continuous Integration
Data Cleansing
Information Engineering
ETL
Distributed Computing Environment
Amazon DynamoDB
PostgreSQL
MongoDB
MySQL
NoSQL
Online Transaction Processing
Oracle Applications
SQL Databases
Data Streaming
Enterprise Data Management
Data Processing
Data Ingestion
Delivery Pipeline
Spark
Gitlab
Data Lake
Information Technology
Apache Flink
Kafka
Data Pipelines
Jenkins

Job description

  • Design and build high performance, scalable data engineering solutions that meet needs of millions of Anywhere's independent agents, brokers, as well as their home buyers and sellers;
  • Design and develop data ingestion pipelines for batch and real-time streaming of data from in-house OLTP systems and third-party data;
  • Collaborate with other Data Engineers at Anywhere for the build out of Next Generation Data Ingestion and Data Enrichment Pipelines;
  • Work with Data Engineering Team to design and develop a Data Lake to store and process terabytes of industry data;
  • Design and develop ETL pipelines to process data in Data Lake for use with APIs, reporting, analytics and related AI and data science;
  • Develop ETL data pipelines to build Enterprise Data Models for Property, Agent, Broker, office and other master partners and related clients and entities;
  • Design and develop CI/CD process for continuous delivery in AWS Cloud;
  • Design, develop, and test robust, scalable data platform components; and
  • Partner with a variety of data and delivery teams within the Anywhere organization, including product engineers to understand their data pipeline needs to assist them with coming up with innovative data engineering solutions.

Requirements

  • Must have a Bachelor's degree in Computer Science, Information Technology or related field, plus five (5) years of experience in the job offered or in any occupation that includes the required experience and skills. Experience must include:
  • 5 years of experience working with Scala or Java;
  • 5 years of experience designing Real-Time Streaming Data Pipelines utilizing Apache Spark or Apache Flink and Apache Kafka;
  • 5 years of development experience in Apache Spark for distributed data processing;
  • 5 years of experience with AWS services including EC2, ECS, S3, EMR and Lambda;
  • 5 years of experience in SQL databases, including PostgreSQL, MySQL, or Oracle, and NoSQL databases, including MongoDB or DynamoDB;
  • 5 years of experience with Continuous Integration & Deployment (CI/CD) processes utilizing tools including AWS CodePipeline, Gitlab, or Jenkins; and
  • 5 years of experience in serving in a lead role for onshore and offshore development teams, providing code reviews, architectural guidance, and technical leadership.

Apply for this position