Lead Data Engineer

• Data Platform & Near Real-Time Analytics Role Overview We are seeking a Lead Data Engineer to design and build a scalable, high-quality data platform that ingests data from multiple sources, ensures data quality and governance, and delivers near real-time insights (15-minute SLA) through Power BI / Microsoft Fabric dashboards.
• This role will provide technical leadership and drive end-to-end data engineering architecture and delivery., * Architecture & Platform Design Design and implement end-to-end data platform architecture (ingestion processing storage serving)
• Define batch and near real-time data pipelines ensuring low-latency and high reliability Make technology decisions across Databricks, Delta Lake, Snowflake, and Azure ecosystem
• Data Engineering & Pipelines Build scalable pipelines using: Databricks / Spark for large-scale processing Delta Lake for ACID-compliant data storage Snowflake for data warehousing and analytics Implement ETL/ELT pipelines with strong data modeling practices Streaming & Real-Time Processing
• Design and implement real-time pipelines using Kafka / Azure Event Hubs Ensure data freshness within ~15-minute SLA
• Enable incremental processing and efficient data updates Data Quality & Governance Establish data quality frameworks (validation, completeness, consistency checks) Implement monitoring, alerting, and data observability
• Define and enforce data governance, lineage, and metadata standards Data Serving & Analytics Enable optimized data layers for Power BI / Microsoft Fabric dashboards Design semantic models and curated data layers for business consumption
• Ensure consistent, accurate, and high-performance reporting Performance & Scalability Optimize pipelines and storage for large-scale datasets (TB/PB)
• Ensure low-latency query performance and efficient compute usage Implement partitioning, indexing, caching, and optimization strategies Leadership & Collaboration Lead and mentor a team of data engineers Collaborate with Technical Product Managers, BI teams, and business stakeholders Drive best practices in coding, architecture, and delivery Manage technical risks, dependencies, and roadmap execution

• Strong experience in data engineering and platform architecture (Lead level)
• Expertise in: Databricks, Spark, Delta Lake Snowflake or similar cloud data warehouses Hands-on with streaming technologies (Kafka / Event Hubs)
• Strong knowledge of data modeling, ETL/ELT, and pipeline design Experience with data quality frameworks and monitoring tools
• Familiarity with Power BI / Microsoft Fabric Strong programming skills (Python, SQL) Experience with Azure ecosystem (ADF, ADLS, AKS - preferred)
• Nice to Have Experience with real-time analytics platforms Exposure to data governance / MDM frameworks
• Familiarity with CI/CD and DevOps practices for data platforms

Key Expectations:

• Own and deliver a robust, scalable data platform
• Ensure high data quality and near real-time availability (15 min SLA) Drive standardization, reusability, and performance optimization
• Enable business-ready, trusted data for analytics and decision-making
• Business Impact Build and scale a modern data platform that delivers trusted, near real-time insights, enabling faster decisions and powering analytics across the organization.

TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client provider of digital technology and transformation, information technology and services, TekWissen® Group is an equal opportunity employer supporting workforce diversity.