Senior Data Engineer
Role details
Job location
Tech stack
Job description
The Senior Data Engineer is a data engineer who can sit with HR business stakeholders to understand their objectives, decompose those needs into a clear technical plan, and then execute that plan end-to-end - from identifying the authoritative source system all the way through to the semantic layer in Microsoft Fabric and the visualizations that put insight in front of the business. This is a hands-on senior individual contributor role for someone who can speak fluently with both a HR business executive and a platform architect, and who can personally build everything in between: ingestion, data contracts, modeling, transformation, testing, semantic layer, and BI. The Senior Data Engineer must be located in the United States. What this Role Looks Like End-To-End:
- Sit with HR business partners to understand the business objective, the decision the data needs to support, and the questions the business is trying to answer.
- Decompose the business need into a technical solution design - identifying the authoritative source of record, the data contract required from that source, the modeling approach, the storage pattern, the transformation logic, the semantic layer design, and the visualization that closes the loop.
- Identify the authoritative source system for each required data element and confirm fit for purpose with the source-system owner.
- Land raw data into the unified ingestion framework using standard ingestion patterns (batch, CDC, streaming, API) as appropriate to the source.
- Define and document data contracts between source systems and the analytics platform, including schema, SLAs, refresh cadence, and change protocols.
- Design and implement conceptual, logical, and physical data models for the domain, aligned to the enterprise business glossary.
- Build production-grade pipelines in Databricks (PySpark, Spark SQL, Delta Lake) and/or Snowflake (including Iceberg tables) depending on the storage pattern that best fits the data product.
- Implement data quality, validation, and testing across the pipeline - unit tests, integration tests, data quality checks, reconciliation, and anomaly alerting.
- Build the semantic layer in Microsoft Fabric (Fabric IQ, OneLake-backed semantic models) so business consumers see business-meaningful entities, measures, and definitions - not raw tables.
- Build and certify the consuming visualizations in Power BI (or comparable BI), then partner with the business on adoption, feedback, and iteration.
- Own the data product after launch - monitoring, KTLO, incident response, SLA adherence, and continuous improvement.
How the Team Works together?
- The HR Analytics engineering team operates as a single team across the US and India, with clear layers and a shared way of working:
- The HR Analytics engineering team is structured in three layers that work together: an Information Architect who owns end-to-end architecture and engineering standards; Principal Data Engineers who set the engineering bar and lead the data engineering team; and Senior Data Engineers, Data Engineers, and Associate Data Engineers who deliver data products across the full vertical.
- The team operates across the US (Frisco, TX) and India (Hyderabad) and US locations in PST Timezone, with engineers in all regions partnering across the timezone gap to keep delivery moving and to share ownership of data products end-to-end.
- Business engagement happens through the team's senior engineering and architecture leadership, with broader engineering team participation that grows over time as engineers build domain context and earn trust with stakeholders.
- Work flows from a business need into a technical solution design, then into a build that spans ingestion through the unified framework, data modeling, transformation across Snowflake and Databricks, semantic layer in Microsoft Fabric, and visualization in Power BI - with quality, testing, documentation, and reliability owned across the whole vertical.
- The team operates a DevOps model - engineers own their data products in production, share an on-call schedule, and rotate the operations role across the team.
- This role is a senior delivery engineer who owns data products end-to-end and partners closely with India-based engineers across the timezone gap to keep delivery moving around the clock.
Core Responsibilities:
- Partner directly with HR business stakeholders to elicit objectives and translate them into solution designs with clear technical decomposition from source to visualization.
- Identify authoritative source systems for each data element and negotiate data contracts with source-system owners, including schema, SLAs, refresh cadence, and change management.
- Land source data into the team's unified ingestion framework using the appropriate ingestion pattern (batch, CDC, streaming, API).
- Design and implement conceptual, logical, and physical data models, including standard data definitions and business glossary alignment for the HR domain.
- Build production-grade pipelines and data products across Snowflake (including Iceberg tables) and Databricks (Delta Lake, Unity Catalog), choosing the storage and compute pattern that fits the use case.
- Apply medallion (bronze, silver, gold) architecture and the team's engineering standards to all data product builds, including naming conventions, documentation, and code review practices.
- Design and build the semantic layer in Microsoft Fabric (Fabric IQ, OneLake) so that business consumers interact with certified, business-meaningful models.
- Design and build Power BI visualizations and reports that close the loop with the business, working iteratively with HR partners on adoption and refinement.
- Implement comprehensive testing across the pipeline - unit tests, integration tests, data quality checks, reconciliation logic, and SLA-driven alerting.
- Own pipeline KTLO (Keep the Lights On) for delivered data products, including monitoring, incident response, and ongoing reliability improvements.
- Write and maintain comprehensive documentation including source-to-target mappings, data lineage, data dictionaries, SLA definitions, semantic model definitions, and runbooks.
- Contribute to and uphold the team's DevOps practices - Git, CI/CD, automated testing, code review.
- Mentor mid-level and junior engineers, share knowledge through demos and training sessions, and raise the engineering standard of the team around you., Role: Senior Data Engineer- US Citizen Location: Minneapolis, MN Rate: Depends on experience Additional Requirements Must be a U.S. Citizen * Must be currently located in Min…, Role: Senior Data Engineer Remote Work: INDIA *Only Consultants local to INDIA are eligible. *No visa Sponsorship Primary Responsibilities: Design, develop, and maintain …
- 2 months ago
Requirements
- Bachelor's degree in Computer Science, Software Engineering, Information Management, or equivalent experience in field - plus 7+ years of related work experience.
- Must be located in the United States.
- 7+ years of hands-on data engineering experience delivering production data pipelines and data products in large enterprise environments.
- Demonstrated ability to sit with business stakeholders, understand their objectives, and personally decompose those objectives into a technical solution design spanning ingestion, modeling, storage, semantic layer, and visualization.
- Expert proficiency in SQL and Python, including PySpark and Spark SQL for distributed data transformation.
- Hands-on experience with Databricks including Delta Lake, Unity Catalog, and workflow orchestration.
- Hands-on experience with Snowflake at production scale, including experience with Iceberg tables and modern open table formats.
- Hands-on experience with Microsoft Fabric including OneLake and Fabric IQ semantic layer design, and a track record of publishing certified data products for downstream consumption.
- Hands-on experience building data visualizations and reports in Power BI, including semantic model design that bridges Fabric models to BI consumption.
- Experience landing data through a unified ingestion framework and defining data contracts with source-system owners, including schema, SLAs, and change protocols.
- Strong data modeling skills - conceptual, logical, and physical - including dimensional modeling, normalized modeling, and modern lakehouse modeling patterns.
- Experience implementing data quality frameworks and pipeline testing, including unit tests, integration tests, data quality checks, and reconciliation.
- Experience with DevOps practices for data pipelines - Git, CI/CD, and automated testing.
- Excellent communication skills - able to convey technical concepts to business stakeholders and translate business objectives into technical requirements.
- Strong problem-solving skills and the ability to operate independently on complex technical problems in ambiguous, high-pressure environments.
Key Skills: SQL, Python, PySpark, Spark SQL, Databricks, Snowflake, Fabric, PowerBI