Azure Data Engineer

Page Michael International Inc
Woodbridge Township, NJ, United States
4 days ago
Apply on www.michaelpage.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours

Tech stack

Sql Data Warehouse Airflow Data Analysis Microsoft Azure Big Data BigQuery Cloud Computing Cloud Database Code Review Data Architecture Information Engineering Data Governance
+25 more
Data Integration Extract Transform Load (ETL) Data Systems Data Warehousing Dimensional Modeling Python (Programming Language) Enterprise Messaging Systems Microsoft SQL Server Online Analytical Processing Online Transaction Processing Cloud Services Standard Sql SQL Stored Procedures Unstructured Data Data Processing Freeform SQL Cloud Platform System Azure Data Factory Snowflake Apache Spark Data Lakes Pyspark Data Management Data Pipelines Databricks

Job description

We’re seeking a Cloud Data Engineer to support a financial services team in Metropark, NJ, focused on developing and optimizing scalable data pipelines and Azure-based cloud solutions. The ideal candidate will have 5+ years of experience with Python, SQL Server, large-scale data processing, and Databricks PySpark, and be available to work onsite three days per week.

  • Build and maintain data pipelines to collect, transform, and load data from multiple sources while ensuring data accuracy and quality.
  • Develop and deploy Spark applications in Databricks to process large-scale datasets.
  • Work with Delta Lake, data warehousing, cloud platforms, data integration, and data modeling solutions.
  • Write and maintain Python and SQL code to support data engineering and analytics initiatives.
  • Design and support dimensional data models for enterprise data warehouse environments.
  • Develop data solutions that leverage real-time and event-driven data processing technologies.
  • Manage and process structured, semi-structured, and unstructured data across various platforms.
  • Optimize Databricks workloads to improve performance, efficiency, and scalability.
  • Troubleshoot Databricks jobs and resolve production issues, errors, and performance bottlenecks.
  • Apply data governance, security, and data management best practices across the data environment.
  • Design, build, and enhance enterprise data warehouse solutions.
  • Create complex SQL queries, stored procedures, and enhancements to existing data processes.
  • Participate in code reviews to ensure quality, performance, and adherence to development standards.

Requirements

We’re seeking a Cloud Data Engineer to support a financial services team in Metropark, NJ, focused on developing and optimizing scalable data pipelines and Azure-based cloud solutions. The ideal candidate will have 5+ years of experience with Python, SQL Server, large-scale data processing, and Databricks PySpark, and be available to work onsite three days per week., * 5+ years of experience developing solutions with Python.

  • 5+ years of experience working with SQL Server and large-scale datasets.
  • 5+ years of experience building and deploying ETL pipelines using Databricks and PySpark.
  • Hands-on experience with cloud data warehouse platforms such as Snowflake, Synapse, Redshift, or BigQuery.
  • Strong understanding of data warehousing concepts, including OLTP, OLAP, dimensional modeling, fact tables, and dimension tables.
  • Experience supporting or leading cloud data platform migration initiatives.
  • Strong data architecture and solution design skills.
  • Experience working with cloud-based data, analytics, and messaging platforms.
  • Cloud certification(s).
  • Experience with workflow orchestration tools such as Airflow.

Benefits & conditions

  • Long term contract with competitive rate and opportunity for conversion!

About the company

A Financial Sservices firm providing corporate banking, investment banking, trade finance, foreign exchange, and treasury services.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.michaelpage.com
Prepare application

Good distractions

Loading talks and stories from around this role…