Data Engineer
Role details
Job location
Tech stack
Job description
As a Data Engineer, you will contribute to a team that supports the data infrastructure of a large organization's digital platform. Your responsibilities will include the design, construction, and maintenance of data pipelines that transport and transform health plan data within Azure-based systems. This role ensures data is delivered with accuracy and timeliness for use in APIs, analytics, and member-facing applications. You will work with SQL Server, Azure Data Factory, Cosmos DB, and Python in a healthcare setting where data quality and compliance are priorities., * Build and maintain ETL/ELT pipelines using Azure Data Factory and SSIS.
- Write T-SQL and Python to extract, transform, and load data across SQL Server, Azure SQL, and Cosmos DB.
- Implement delta loading, deduplication, and idempotency patterns for reliable pipeline execution.
- Design Slowly Changing Dimension (SCD) logic to track member and plan data history.
- Apply data quality checks and validation rules during ingestion and transformation.
- Monitor pipeline runs, investigate failures, and implement observability into data flows.
- Follow Git and Azure DevOps workflows, including branching, pull requests, and CI/CD pipeline integration.
- Document data flows, field mappings, and transformation logic.
- Handle healthcare data in compliance with HIPAA requirements.
Requirements
Experience: 2-5 years in data engineering, ETL development, or a related data-focused role. Demonstrated experience with Git, including branching, commits, and pull request workflows.
Technical Skills: Hands-on experience writing T-SQL queries (SELECT, JOINs, WHERE filters, aggregations). Experience building or maintaining ETL pipelines using SSIS or Azure Data Factory (ADF). Experience with SQL Server or Azure SQL as a primary data store and experience working in cloud technologies such as Azure, AWS, or Google Cloud Platform.
Professional Competencies: Proficiency in T-SQL for data extraction, transformation, and validation. Understanding of core data engineering concepts such as delta loading, deduplication, and idempotency. Familiarity with development tools including SSMS, Visual Studio, VS Code, and the Azure Portal. Knowledge of the data pipeline lifecycle, data validation, and source-to-target mapping is required. Familiarity with Agile/Scrum methodology and knowledge of HIPAA requirements are necessary., * Experience with Python for data transformation and pipeline scripting, including libraries like pandas.
- Familiarity with Azure Cosmos DB or another NoSQL data store.
- Understanding of delta loading, deduplication, and idempotency patterns in data pipelines.
- Exposure to bulk load strategies and partitioning for large data volumes.
- Experience integrating pipelines into Azure DevOps CI/CD workflows.
- Exposure to healthcare data structures (member, claims, eligibility).
- Familiarity with Azure Functions, Blob Storage, and workflow orchestration tools like Apache Airflow or Azure Logic Apps.
- Exposure to Azure OpenAI or AI/ML integration concepts.
Benefits & conditions
The pay rate for this position is between $55.00 and $60.00 per hour. A comprehensive benefits package is available to eligible employees.