> Markdown version of [/jobs/ext/211800-data-engineer](https://www.wearedevelopers.com/jobs/ext/211800-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** BLDG SVC 32 B-J - **Location:** New York, NY, United States - **Experience:** Starter - **Salary:** $100,000.0 - $115,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Data Analysis, Automation of Tests, Microsoft Azure, Big Data, Cloud Database, Information Systems, Continuous Integration, Data Architecture, Data Dictionary, Information Engineering, Data Governance, Data Integration, Data Integrity, Extract Transform Load (ETL), Data Transformation, Data Security, Data Warehousing, Software Debugging, DevOps, Python (Programming Language), Microsoft Dynamics, Microsoft SQL Server, Modular Design, Query Optimization, SQL Stored Procedures, SQL Databases, Enterprise Data Management, Data Processing, Data Storage Technologies, Cloud Platform System, Azure Data Factory, Database Optimization, Software Troubleshooting, Technical Debt, Data Layers, Data Lakes, Information Technology, Performance Monitor, Data Management, Tools for Reporting, Cloud Migration, Software Version Control, Data Pipelines, Databricks - **Published:** May 22, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=d72bea8288f357ee ## About the Role Do you have experience in Version control systems?, * 1-3 years of professional experience in data engineering, data integration, data warehousing, analytics engineering, or related technical disciplines * Demonstrated ability to support scalable data pipelines, data transformation processes, and cloud-based data initiatives within collaborative technical environments * Strong understanding of software engineering best practices applied to data engineering, including modular design, automated testing, CI/CD, version control, and idempotent processing * Advanced SQL development skills, including stored procedures, functions, triggers, query optimization, indexing strategies, and performance troubleshooting across large-scale datasets * Strong understanding of modern data engineering concepts and architecture, including ETL/ELT frameworks, pipeline orchestration, data modeling, schema evolution, batch ingestion patterns, and medallion architecture principles within Databricks/Delta Lake environments * Proficiency in Python (preferred) for developing scalable, maintainable, and production-ready data pipelines within Databricks * Experience working within the Azure ecosystem, especially Azure Databricks, Delta Lake, and cloud migration initiatives * Experience working within Agile/DevOps delivery environments and cross-functional technical teams * Prior experience working with healthcare claims data and understanding healthcare/benefits domain concepts, including claims, eligibility, providers, and benefit fund cost drivers Soft Skills (Interpersonal Skills): * Ability to communicate technical concepts and tradeoffs effectively to both technical and non-technical stakeholders * Strong collaboration and participation within Agile/DevOps teams * Comfort working in ambiguous, fast-paced, and evolving environments * Ownership mentality with proactive identification of risks, inefficiencies, and improvement opportunities * Pragmatic decision-making and ability to balance ideal architecture with delivery timelines * Continuous learning mindset and adaptability to new technologies and platforms * Mission-oriented approach focused on improving operational efficiency and member outcomes Education: Bachelor's degree in Computer Science, Information Systems, Data Science, Engineering, or a related technical field, or equivalent combination of education and hands-on experience Language Skills: Strong verbal and written communication skills in English, including the ability to read, write, understand, and effectively communicate technical and business information. ## Description As a Data Engineer you will get to play a key and a collaborative role in the delivery of powerful data-driven products that support 32BJ Health Fund's mission of providing high-quality and low-cost healthcare to its union members. The Data Engineer will be responsible for providing internal analysts with accurate datasets by implementing best practices in data collection, movement, storage, and transformation of large datasets. This individual will work with both current ETL/Data Warehousing and provide direction for future development of data storage, streaming and pipeline architectures. Essential Duties and Responsibilities: * Works with Health Fund Analytics, Operations, and IT to design, develop, maintain, and optimize complex data pipelines supporting both on-premises and Azure cloud environments * Migrates and integrates data from disparate internal and external sources into centralized Azure cloud and on-premises data warehouse solutions using established data engineering best practices * Uses SQL, Azure Data Factory, Databricks, Python, and other data transformation tools to develop and automate scalable ETL/ELT processes for ingesting, transforming, and loading data from multiple vendors into centralized data platforms * Designs and implements resilient ingestion pipelines capable of handling schema drift, missing or invalid fields, inconsistent vendor formats, and evolving source system structures * Builds scalable, flexible, and extensible data models that support evolving business requirements, onboarding of new vendors, and downstream analytics/reporting needs * Implements and maintains medallion architecture principles with clear separation of raw, refined, and curated data layers * Diagnoses and resolves performance bottlenecks impacting pipeline efficiency, reporting processes, and downstream data consumers across SQL Server and Databricks environments * Supports and optimizes data workflows across hybrid on-premises and cloud-based platforms during ongoing cloud migration initiatives * Translates operational and business requirements into scalable, maintainable, and efficient data engineering solutions * Anticipates and mitigates risks related to vendor data variability, schema evolution, and data quality issues to ensure data reliability and continuity * Prioritizes and manages technical debt to improve platform stability, maintainability, scalability, and delivery efficiency * Generates data subsets, semantic models, APIs, variables, and reusable datasets required for integration with internal applications, analytics tools, and public-facing platforms * Works collaboratively with IT and Operations teams to evaluate, implement, and support scalable cloud-based solutions, including Azure and Dynamics 365 technologies * Supports implementation and ongoing maintenance of enterprise Data Governance policies, standards, and data management best practices within assigned domains * Supports data engineering operations through proactive monitoring, alerting, troubleshooting, debugging, and maintenance activities to minimize downtime and ensure data quality * Interfaces with internal stakeholders and external vendor IT teams to resolve data quality issues and ensure HIPAA-compliant data handling, transfer, and storage practices * Creates and maintains clear technical documentation, including data dictionaries, schemas, user guides, quick-start materials, and workflow/process documentation * Provides training sessions, tutorials, and ongoing support to analysts and business users on data access, query development, reporting tools, and available data resources * Serves as a subject matter expert on internal and external data sources, data architecture, and enterprise data management practices, The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals to perform the essential functions. The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals to perform the essential functions. * Under 1/3 of the time: Standing, Walking, Climbing or Balancing, Stooping, Kneeling, Crouching, or Crawling * Over 2/3 of the time: Talking or Hearing * 100% of the time: Using Hands ## Related Videos - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)