DS
Dhannush Subramani
Top Big Data Technologies That You Need to Know
about 4 years ago
Data Engineer
Parkar Consulting Group LLC
New York, NY, United States
23 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Working hours
Regular working hours
Job source
Tech stack
Airflow
Microsoft Azure
Big Data
Cloud Computing
Continuous Integration
Information Engineering
Data Governance
Data Infrastructure
Data Systems
Apache Hive
Python (Programming Language)
SQL Databases
+4 more
Workflow Management Systems
Pyspark
Data Management
Databricks
Requirements
- Minimum 5 to 7 years of data engineering experience, with at least 3+ years of hands-on Databricks experience in production environments.
-
Migration Expertise Proven experience in migrating on-premises data systems to cloud platforms.
-
Technical Skills Strong proficiency in Python, PySpark, and SQL. Experience in converting legacy SQL dialects to Spark SQL is a plus.
-
Cloud Knowledge Familiarity with cloud computing concepts, preferably within Azure ecosystems. Experience integrating cloud functions or runtime services with data platforms is desirable.
- Best Practices & Tools Solid understanding of CI/CD, data quality frameworks, and orchestration tools such as Databricks Workflows, Airflow, or dbt.
Preferred Skills
- Experience with large-scale data modernization programs
- Exposure to financial services or regulated data environments
- Strong problem-solving and stakeholder communication skills
- Investment banking background would be added advantage.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlists2:15 min
Empowering domain teams with an open data platform
Sandhya Menon · World Congress 2026 Europe
3:28 min
Defining big data and machine learning fundamentals
Ayon Roy · LIVE
3:03 min
Career evolution in data engineering and AI platforms
Maria Apazoglou · Coffee With Developers
4:32 min
Harnessing Spark with Python using PySpark and Py4J
Ayon Roy · LIVE
2:10 min
Why organizations combine big data and machine learning
Ayon Roy · LIVE
2:34 min
Capabilities of the Apache Spark processing engine
Ayon Roy · LIVE
Videos
See allRelated articles
See all
EM
Eli McGarvie
Data Engineer Salary UK
about 3 years ago
.gif)
BB
Benedikt Bischof
Making Data Warehouses Fast: A Developer’s Story
about 4 years ago
AJ
Austin Joy
What Are The Top Skills Required For Azure Developers?
over 4 years ago
EM
Eli McGarvie
Data Analyst Salary in the UK
about 3 years ago
EM
Eli McGarvie
Highest Paying Tech Companies for Developers
over 3 years ago