Data Engineer

Wise Skulls llc
New York, NY, United States
14 days ago
Apply on www.thejobnetwork.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
6 years minimum
Working hours
Regular working hours

Tech stack

Unity 3d Airflow Amazon S3 Continuous Integration Data Architecture Information Engineering Data Transformation Distributed Computing Environment Python (Programming Language) Performance Tuning Software Deployment SQL Databases
+8 more
Data Streaming Workflow Management Systems Data Processing Git Data Lakes Pyspark Data Pipelines Databricks

Requirements

  • 6+ years of data engineering experience with substantial hands-on Databricks expertise \n

  • Strong proficiency in SQL, Python, and PySpark \n

  • Experience designing and developing scalable data pipelines using Medallion Architecture \n

  • Strong understanding of Databricks Lakehouse architecture, Delta Lake, and Unity Catalog \n

  • Experience with batch and incremental data processing, workflow orchestration, and data transformation \n

  • Expertise in SQL query optimization, distributed processing, and performance tuning \n

  • Experience with data quality, observability, lineage, and production troubleshooting \n

  • Familiarity with Git, CI/CD, and production deployment practices \n

  • Experience with AWS services, particularly S3, is a plus \n

  • Knowledge of Auto Loader, Structured Streaming, CDC, Databricks Asset Bundles, or Apache Airflow is preferred \n

  • Databricks Data Engineer Associate or Professional certification is a plus \n

  • Experience with streaming, clickstream, or advertising datasets is an added advantage \n

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.thejobnetwork.com
Prepare application

Good distractions

Loading talks and stories from around this role…