Jr. Data Engineer
Role details
Job location
Tech stack
Job description
The client is strategically transitioning the Data Warehouse engineering operations onshore and is looking for a driven Data Warehouse Engineer to help lead this initiative. Your primary focus will be migrating, building, and optimizing ELT pipelines that extract data from third-party APIs and load it into our data lake utilizing managed Delta tables. You will initially work closely with our existing offshore team to complete a comprehensive Knowledge Transfer (KT)., ELT Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines to migrate data from various third-party API sources into our enterprise data lake. Databricks Ecosystem: Create, manage, and optimize managed Delta tables within a Databricks environment to ensure high data reliability and query performance. API Integration: Securely connect to, authenticate with, and extract data from a wide variety of third-party RESTful APIs. Data Modeling & Querying: Write clean, scalable, and efficient Python and SQL code to transform raw data into analytics-ready datasets. Knowledge Transfer (KT): Actively participate in onboarding and KT sessions with the current offshore team to seamlessly transition ownership of existing pipelines and architecture. Documentation: Maintain clear technical documentation for data pipelines, API integrations, and Delta table schemas to support future onshore hires.
Requirements
Experience: 4+ years of experience in data engineering, data warehousing, or a similar technical role. Core Languages: Strong, hands-on proficiency in Python and SQL. Databricks & Data Lakes: Practical experience working with Databricks, specifically building and maintaining Delta tables and working within a modern data lake architecture. API Expertise: Proven experience extracting and handling data from third-party APIs (handling pagination, rate limiting, JSON/XML parsing, etc.). Methodology: Solid understanding of ELT methodologies and cloud-based data architecture. Communication: Excellent verbal and written communication skills, with the ability to effectively absorb complex technical information during cross-border knowledge transfers.
Benefits & conditions
- $65,000-90,000 per year