WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Databricks Architect - Tarrytown, NY [Hybrid]

OMG Technologies
Sleepy Hollow, NY, United States
11 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
4 years minimum
Working hours
Regular working hours
Job source
www.dice.com

Tech stack

Unity 3d Artificial Intelligence Airflow Amazon Web Services Amazon S3 Data Architecture Data Governance Extract Transform Load (ETL) Apache Hive Identity and Access Management Python (Programming Language) Data Streaming
+7 more
Data Lakes Pyspark Data Analytics Real Time Data Apache Nifi Serverless Computing Databricks

Job description

  • Lead enterprise-scale Databricks Lakehouse architecture and implementation on AWS.
  • Drive migration of EMR, NiFi, Redshift, and legacy ETL workloads to Databricks.
  • Evaluate newer Databricks capabilities, including Serverless, Lakehouse Federation, Genie, Databricks Apps, Lakebase, and AI/ML.
  • Architect scalable batch, streaming, CDC, and real-time data solutions.
  • Design reusable frameworks using Delta Lake, PySpark, Auto Loader, DLT, and Databricks Workflows.
  • Define governance, security, lineage, and data-quality standards using Unity Catalog.
  • Establish best practices for performance, scalability, and cost optimization.
  • Lead customer workshops, architecture reviews, and solution assessments.
  • Define target-state architecture, migration roadmaps, and reusable patterns.

Requirements

  • 15+ years of overall Data & Analytics experience.
  • 4+ years of hands-on Databricks experience as specified in the JD.
  • Strong experience with Databricks, Delta Lake, Unity Catalog, DLT, and Auto Loader.
  • Proficiency in PySpark, Spark SQL, Python, and Structured Streaming.
  • AWS experience with S3, IAM, Glue, Redshift, EMR, and Lambda.
  • Experience with Airflow, CDC, and enterprise data migration.
  • Knowledge of data governance, security, and observability.
  • Strong customer-facing, solution architecture, and technical leadership skills., * Healthcare or pharmaceutical domain experience is preferred but not mandatory.
  • Experience with enterprise-scale data platform modernization and migration is beneficial.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…