Azure Data Engineer Lead
Role details
Job location
Tech stack
Job description
Unified Analytics Platform: Work across Fabric workloads: COSMSO, Kusto, Data Factory (ETL), Synapse Data Engineering (Spark/SQL), Synapse Data Warehousing (SQL), and OneLake for storage.
Data Loading & Architecture: Design data loading patterns, Lakehouse architectures, and orchestration processes for enterprise-scale analytics.
Data Preparation & Enrichment: Prepare and enrich data for analysis, ensuring data quality and consistency checks.
Implement semantic models for BI and self-service analytics.
Security & Compliance: Secure and manage analytics assets, enforce governance policies, and monitor compliance and Managed Identity.
Performance & Optimization: Monitor and optimize Data pipelines and workloads for cost efficiency and high availability.
Collaboration & Stakeholder Engagement, Offshore team collaboration
Work closely with architects, analysts, and business teams to translate requirements into scalable solutions.
Enable Power BI integration for visualization and reporting.
Data Integration & Orchestration: Build and manage data pipelines using COSMOS, Azure Data Factory (ADF), Synapse, Fabric pipelines for batch and incremental ingestion. Automate ETL workflows and implement parameterized pipelines for scalability.
Data Transformation & Modelling: Develop ETL code / pipelines to clean, transform, and convert raw files into optimized formats (Parquet/Delta). Create views, stored procedures, and dimensional models for reporting and analytics.
Performance Optimization: Monitor Dynamic Management Views (DMVs) in Synapse to analyze system health and reduce bottlenecks. Implement workload management strategies for query performance and resource governance.
Security & Governance: Apply role-based authentication, encryption/decryption logic, and compliance controls.
Automate governance tasks to reduce manual intervention.
Requirements
Synapse; SQL Server; Python; Azure Storage; ADF; Cosmos; Kusto; Digital: PySpark; Fabric; Data Engineering; Data Warehouse, Reliability & Monitoring: Build failure monitoring systems integrated with Logic Apps for alerting and proactive issue resolution.
Strong in Azure Data Services ADF, Azure SQL database, Synapse
Ability to understand vast amounts of data, identify and fix data issues.
Strong in database and DW concepts, Excellent technical & analytical skills with strong business acumen
Strong communication skills
Must have Microsoft Azure Cloud and services experience.
Benefits & conditions
Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support: Maternal & Parental Leaves.
Insurance Options: Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
Time Off: Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
