Sr. AWS Data Engineer (Redshift/ S3/ Databricks)

Resource 1 is in need of a Data Engineer (AWS/ SQL/ Databricks) with at least 5 years of experience for a 6-month contract-for-hire with our client in the north suburbs. Individual will need to be local and able to work onsite in Northfield, IL 3 days per week. The resource will focus on the company’s AWS-based data platform, primarily developing and stabilizing ETL pipelines and dimensional models in Redshift and Databricks to enable analytics and Tableau reporting. They will investigate and resolve data breaks, design and implement data transformations, and support planning and budgeting application integrations., * Develop and maintain data and cloud solutions leveraging Databricks and AWS technologies.

• Develop/ maintain ETL pipelines to ingest and process data into Redshift data warehouse from various data sources, including Salesforce and Dynamics 365.
• Build, schedule, and manage data ingestion jobs using AWS Glue, Airflow, S3, and related services.
• Write and maintain complex SQL stored procedures and other database code for data manipulation.
• Design, build, and maintain data pipelines across all layers of the Medallion Architecture (Bronze, Silver and Gold).
• Design and implement dimensional data models and data marts for analytics and Tableau consumption.
• Collaborate with the Analytics & Insights team and business stakeholders to translate reporting requirements into data models and metadata to support operational reporting needs.
• Support planning and budgeting application interfaces and related Python-based validation scripts.
• Apply strong problem-solving and critical-thinking skills to troubleshoot complex technical issues and identify opportunities for improvement.

• 5+ years of database development/ data engineering experience.
• Proficiency with AWS data services such as Redshift, S3, Glue, and Airflow scheduling.
• Strong SQL skills including complex queries and stored procedures.
• Experience with data modeling and dimensional modeling best practices.
• Python programming experience and ability to write and maintain notebooks or scripts.
• Familiarity with Databricks or ability to learn Databricks notebooks and workflows quickly.
• Strong data analysis and problem-solving skills.

Plusses:

• Hands-on Databricks experience and familiarity with PySpark.
• Experience with Dynamics 365 (D365) tables and integrations.

Technical Environment:

• AWS Redshift, Databricks, PySpark, Python, SQL Stored Procedures, EDW (Enterprise Data Warehouse), AWS S3, AWs Glue, Tableau, Power BI, PostgreSQL, ETL, Dynamics 365 (D365), ServiceNow and Jitterbit.