Databricks with python

Siri InfoSolutions Inc
Jersey City, NJ, United States
2 months ago
Apply on www.careerjet.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$170,000.0 - $200,000.0
Working hours
Regular working hours

Tech stack

Agile Methodology Airflow Big Data Continuous Integration Information Engineering Extract Transform Load (ETL) Data Transformation Data Warehousing Relational Databases Database Queries Software Debugging Apache Hive
+10 more
Python (Programming Language) Performance Tuning Unstructured Data Workflow Management Systems Git Data Lakes Pyspark Data Lakehouse Data Pipelines Databricks

Requirements

10+ years of experience in Data Engineering, Data Warehousing, or Big Data development. 5+ years of hands-on experience with Databricks, Python, and PySpark. Strong hands-on experience in Databricks, Python, and PySpark development. Experience building and supporting large-scale ETL/ELT data pipelines. Strong knowledge of Spark SQL, Delta Lake, and Data Lakehouse architecture. Experience working with structured and unstructured data from multiple sources. Strong SQL skills and experience with relational databases. Experience with data modeling, data transformation, and data quality processes. Knowledge of orchestration tools such as Airflow, ADF, or similar. Experience with Git, CI/CD, and Agile development methodologies. Strong analytical, debugging, and performance tuning skills. Excellent verbal and written communication skills.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.com
Prepare application

Good distractions

Loading talks and stories from around this role…