Azure Data Engineer
Tech Wish, LLC
New York, NY, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.thejobnetwork.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
6 years minimum
Working hours
Regular working hours
Job source
Tech stack
Microsoft Azure
Big Data
Cloud Computing
Computer Programming
Continuous Integration
Data Architecture
Information Engineering
Data Governance
Data Infrastructure
Data Migration
Data Warehousing
Apache Hive
+18 more
Python (Programming Language)
Performance Tuning
Query Optimization
Software Engineering
Data Streaming
Systems Integration
Cloud Platform System
Azure Data Factory
Apache Spark
Caching
Change Data Capture
Data Lakes
Pyspark
Kubernetes
Information Technology
Stream Processing
Data Pipelines
Databricks
Requirements
- Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline, with 6-10 years of experience in Data Engineering, Application Development, System Integration, or Enterprise Data Warehouse projects.
- Strong hands-on experience designing and implementing modern data platforms on Azure Databricks, leveraging Delta Lake, Delta Live Tables (DLT), and Medallion Architecture (Bronze, Silver, Gold) to build scalable and reliable data pipelines.
- Proven expertise in developing and optimizing batch and streaming data pipelines using PySpark, Spark SQL, and Python, with a strong focus on performance tuning, scalability, and cost optimization.
- Extensive experience in Databricks job performance optimization, including query tuning, partitioning strategies, data skew mitigation, caching, cluster sizing, workload optimization, and Spark execution plan analysis.
- Strong proficiency in SQL programming and data modeling concepts, including relational, dimensional, and lakehouse data architectures.
- Deep understanding of Delta Lake features such as ACID transactions, schema evolution, time travel, change data capture (CDC), and data quality enforcement.
- Experience implementing data quality, monitoring, and orchestration frameworks using Delta Live Tables expectations, Databricks Workflows, and Azure-native services.
- Strong exposure to the full Software Development Life Cycle (SDLC), including design, development, testing, CI/CD, deployment, and production support.
- Solid understanding of Lakehouse architecture, cloud-based analytics platforms, data governance, and data engineering best practices.
- Knowledge of real-time data processing, data integration patterns, and enterprise-scale data migration initiatives is a strong advantage.
- Demonstrated ability to stay current with evolving cloud and big data technologies and contribute effectively to architecture discussions, technology strategy, and platform modernization initiatives.
- Self-motivated, proactive, and capable of working independently in a fast-paced environment while collaborating with globally distributed teams.
- Strong analytical, problem-solving, communication, and stakeholder management skills.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.thejobnetwork.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…