Microsoft Azure Data Engineer Associate

THE JUDGE GROUP, INC.

Phoenix, AZ, United States

3 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

2 years minimum

Compensation

$158,080.0 - $168,480.0

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Artificial Intelligence Airflow Amazon Web Services Apache HTTP Server Microsoft Azure Big Data BigQuery Cloud Computing Cloud Database Cloud Engineering Cloud Storage Information Systems

+46 more

Computer Programming Databases Information Engineering Data Infrastructure Data Transformation Data Migration Data Systems Data Warehousing DevOps Web Development Distributed Systems Apache Hadoop Hadoop Distributed File System Apache Hive Python (Programming Language) Node.Js NoSQL Cloud Services Standard Sql Cloudera Software Engineering SQL Databases Data Streaming Parquet Data Processing Google Cloud Data Storage Technologies Cloud Platform System ReactJS Apache Spark Git Data Lakes Pyspark Kubernetes Information Technology Apache Flink Data Analytics Apache Kafka Spark Streaming Data Management Front End Software Development Data Lakehouse Stream Processing Data Pipelines Docker Jenkins

Job description

We are seeking a highly skilled Senior Data Engineer to design, build, and optimize large-scale cloud-native data platforms and streaming solutions. In this role, you will partner with cross-functional engineering, architecture, and business teams to develop scalable data pipelines, modern lakehouse architectures, and real-time data processing capabilities that support enterprise analytics and AI initiatives.

The ideal candidate has deep expertise in Google Cloud Platform (Google Cloud Platform), big data technologies, streaming frameworks, and modern data engineering practices, along with a strong track record of delivering cloud migration and data modernization programs. Responsibilities

Design, develop, and maintain scalable data pipelines and streaming solutions using modern cloud-native technologies.
Build and support real-time and batch data processing systems using Apache Spark, Kafka, Flink, and Python.
Architect and implement enterprise data lakehouse solutions leveraging industry-standard data storage and governance practices.
Develop and optimize cloud-based data solutions using Google Cloud Platform (Google Cloud Platform) services, including:
BigQuery
Cloud Storage
Dataproc
Cloud Composer
Lead data migration initiatives from on-premises environments to cloud-native architectures.
Design and implement automated data quality, governance, monitoring, and security controls.
Collaborate with architects, engineers, product teams, and stakeholders to deliver scalable and reliable data solutions.
Build and maintain CI/CD pipelines and DevOps processes supporting data platform deployments.
Evaluate and implement emerging technologies to improve data engineering capabilities and operational efficiency.
Contribute to AI-enabled data solutions utilizing modern GenAI frameworks and agent-based architectures.

Requirements

Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent practical experience.
5+ years of professional experience in Data Engineering or related software engineering disciplines.
5+ years of hands-on experience with Hadoop and cloud-based data platforms.
3+ years of experience designing and implementing data lakehouse architectures.
2+ years of hands-on experience developing streaming applications using:
Apache Kafka
Apache Flink
Spark Streaming
Strong programming experience with:
Python
PySpark
SQL
Experience with Google Cloud Platform (Google Cloud Platform) services including BigQuery, Cloud Storage, Dataproc, and Cloud Composer.
Experience with NoSQL technologies, including document, graph, key-value, and columnar databases.
Strong understanding of data warehousing, distributed computing, and cloud data architecture.
Experience with Hadoop ecosystem technologies such as:
Hive
HDFS
Parquet
Apache Iceberg
Delta Lake
Experience implementing scalable, resilient, and highly available data platforms.

Preferred Qualifications

Professional cloud certification such as:
Google Cloud Professional Data Engineer
AWS Specialty Data Analytics
Microsoft Azure Data Engineer Associate
Experience with GenAI frameworks such as LangChain and LangGraph.
Experience with DevOps and CI/CD tools including:
Git
Jenkins
Docker
Kubernetes
Experience developing web applications using React and Node.js.
Strong communication, stakeholder management, and consulting skills.
Experience working in highly collaborative Agile engineering environments.

Key Technologies

Cloud: Google Cloud Platform (Google Cloud Platform), BigQuery, Cloud Storage, Dataproc, Cloud Composer

Data Engineering: Spark, PySpark, Kafka, Flink, Airflow, SQL

Big Data: Hadoop, Hive, HDFS, Parquet, Iceberg, Delta Lake

Databases: NoSQL, Columnar, Graph, Document, Key-Value Stores

DevOps: Git, Jenkins, Docker, Kubernetes

AI/ML: LangChain, LangGraph

Frontend (Nice to Have): React, Node.js

This position offers the opportunity to work on large-scale enterprise data modernization initiatives, cloud transformation programs, and next-generation AI-driven data platforms.