Databricks with python

Siri InfoSolutions Inc
Jersey City, NJ, United States
24 days ago
Apply on www.careerjet.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$170,000.0 - $200,000.0
Working hours
Regular working hours

Tech stack

Agile Methodology Airflow Big Data Continuous Integration Information Engineering Extract Transform Load (ETL) Data Transformation Data Warehousing Relational Databases Database Queries Software Debugging Apache Hive
+10 more
Python (Programming Language) Performance Tuning Unstructured Data Workflow Management Systems Git Data Lakes Pyspark Data Lakehouse Data Pipelines Databricks

Requirements

10+ years of experience in Data Engineering, Data Warehousing, or Big Data development. 5+ years of hands-on experience with Databricks, Python, and PySpark. Strong hands-on experience in Databricks, Python, and PySpark development. Experience building and supporting large-scale ETL/ELT data pipelines. Strong knowledge of Spark SQL, Delta Lake, and Data Lakehouse architecture. Experience working with structured and unstructured data from multiple sources. Strong SQL skills and experience with relational databases. Experience with data modeling, data transformation, and data quality processes. Knowledge of orchestration tools such as Airflow, ADF, or similar. Experience with Git, CI/CD, and Agile development methodologies. Strong analytical, debugging, and performance tuning skills. Excellent verbal and written communication skills.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

1:41 min

Visualizing the complex developer journey for JVM ecosystems

Bobur Umurzokov · LIVE

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

56 sec

Favorite git commands and the importance of patch commits

Eileen Uchitelle Eileen Uchitelle +1 · Coffee With Developers

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · World Congress 2024

Videos

See all

Related articles

See all