Data Architect - Databricks/AWS

CUSTOM STAFFING INC
New York, NY, United States
6 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$208,000.0
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Data Analysis Cloud Database Data Architecture Information Engineering Extract Transform Load (ETL) Data Security Data Warehousing Database Design Identity and Access Management Python (Programming Language) Standard Sql
+12 more
SQL Databases Tableau (Software) Apache Spark Pyspark Gitlab-ci Infrastructure Automation Frameworks Information Technology AWS Data Analytics Data Management Machine Learning Operations Data Pipelines Databricks

Job description

We are seeking an experienced Data Architect to design scalable enterprise data solutions using Databricks and AWS. This role will lead cloud data architecture, data pipelines, governance, modeling and analytics within a complex, highly regulated environment., * Architect Databricks and AWS data platforms for large-scale analytical workloads.

  • Design ETL/ELT pipelines, lakehouse solutions and dimensional data models.
  • Integrate Unity Catalog, AWS IAM and cross-account data access.
  • Develop standards for PySpark, Python, Spark and SQL.
  • Establish data quality, governance, security and access-control frameworks.
  • Optimize compute, storage, clusters and overall platform costs.
  • Implement GitLab CI/CD and Infrastructure as Code practices.
  • Design semantic layers for Tableau and BI reporting.
  • Support MLOps and generative AI initiatives.
  • Partner with technical and business teams and mentor data engineers.

Requirements

  • Databricks and Unity Catalog
  • AWS cloud data architecture
  • PySpark, Python, Spark and SQL
  • ETL/ELT pipelines
  • Data modeling and governance
  • GitLab CI/CD and Infrastructure as Code
  • Tableau and BI architecture, * Bachelor’s degree in computer science or a related field.
  • 8+ years of data architecture, data engineering or database design experience.
  • Extensive Databricks and AWS cloud data architecture experience.
  • Strong PySpark, Python, Spark, SQL and ETL/ELT expertise.
  • Expert knowledge of data modeling, lakehouses, data warehouses and governance.
  • Experience delivering secure, scalable and highly available data platforms.
  • Strong communication and cross-functional leadership skills.
  • Tableau, MLOps and Domino Data Lab experience preferred.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:27 min

Managing traffic and tracking costs with Databricks Unity Catalog

Viktoria Semaan Viktoria Semaan · WWC Europe 2026

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

1:43 min

AWS infrastructure stack and data flow pipeline overview

Artem Volk Artem Volk +1 · WWC 2024

1:59 min

Key takeaways and accessing the Databricks developer toolkit

Viktoria Semaan Viktoria Semaan · WWC Europe 2026

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

3:38 min

Architecting a unified data and machine learning workbench

Kapil Gupta Kapil Gupta · WWC 2025

Videos

See all

Related articles

See all