Lead Data Engineer

We are seeking an experienced Data Engineering Architect / Lead Data Engineer to design, build, and optimize scalable data platforms and pipelines. The ideal candidate will have extensive experience in ETL/ELT development, cloud-based data engineering, real-time and batch data processing, and modern data warehouse architectures. This role requires strong expertise in designing end-to-end data solutions that are scalable, secure, and aligned with enterprise standards., Data Architecture & Solution Design

• Design end-to-end data engineering architectures for enterprise-scale solutions.
• Develop scalable architectures for:
• Data Lakes and Lakehouse platforms
• Enterprise Data Warehouses
• Streaming and real-time data processing systems
• Ensure solutions align with enterprise architecture, security, governance, and compliance standards.
• Review and approve technical designs and implementation strategies.

Data Pipeline Development & Management

• Lead the design and development of scalable ETL/ELT pipelines.
• Build and manage data ingestion pipelines for both batch and real-time data.
• Process structured and semi-structured data efficiently.
• Optimize data pipelines for performance, reliability, scalability, and cost.
• Manage schema evolution, metadata, and pipeline dependencies.

Data Quality, Reliability & Operations

• Establish and enforce data quality standards and validation frameworks.
• Implement monitoring, alerting, logging, and observability for data pipelines.
• Perform root cause analysis and resolve data-related production issues.
• Drive operational excellence by improving system stability and reliability.

DevOps / DataOps

• Build and maintain CI/CD pipelines for data engineering workloads.
• Automate testing, deployment, and rollback processes.
• Improve platform reliability and deployment efficiency through automation and DevOps best practices.

• Advanced expertise in designing and developing ETL/ELT pipelines.
• Strong experience with batch data processing and near real-time/streaming data pipelines.
• Hands-on experience working with structured and semi-structured data.
• Strong knowledge of:
• Incremental data loading
• Change Data Capture (CDC)
• Pipeline orchestration and dependency management
• Strong programming skills in Python (preferred), Scala, or Java.
• Experience optimizing large-scale data processing workloads for performance and cost.
• Solid understanding of data modeling concepts:
• Star Schema
• Snowflake Schema
• Normalized and denormalized data models
• Hands-on experience with at least one major cloud platform:
• Microsoft Azure
• Amazon Web Services (AWS)
• Google Cloud Platform (GCP)
• Strong experience with modern data warehouses such as:
• Snowflake
• Azure Synapse
• Google BigQuery
• Amazon Redshift, * Experience with modern DataOps and CI/CD practices.
• Knowledge of data governance, security, and compliance frameworks.
• Experience designing enterprise-scale cloud-native data platforms.
• Strong analytical, troubleshooting, and communication skills.