> Markdown version of [/jobs/ext/2564909-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2564909-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** BDIPLUS INC.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Salary:** $120,000.0 - $140,000.0
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), IBM AIX, Amazon Web Services, Amazon S3, Apache HTTP Server, IBM System I, COBOL (Programming Language), Computer Programming, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), JSON, Python (Programming Language), Node.Js, Open Database Connectivity, Query Optimization, Standard Sql, Virtual Storage Access Methods, YAML, Openapi, Apache Spark, Caching, Apigee, Fastapi, Debezium, Data Analytics, Data Lakehouse, Api Design, Api Gateway, Data Pipelines, User Identification
- **Published:** August 5, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=0dd6a0af246be898

## About the Role

* 5+ years of data engineering experience in enterprise-scale environments
 * Strong expertise with Denodo (VQL development, query optimization, caching, governance)
 * Experience with Apache Iceberg on AWS (S3, Glue, Athena), including schema evolution and partitioning
 * Hands-on experience with Informatica
 * Experience implementing data quality frameworks
 * Strong SQL and programming skills (Python, Spark; dbt preferred)
 * Experience with CDC technologies (e.g., Debezium, Informatica CDC, or equivalent)
 * Familiarity with API development (FastAPI, Node.js) and OpenAPI standards
 * Experience working with AI-assisted development tools (Claude Code preferred; training available)

Preferred Qualifications

 * Experience with mainframe data environments (COBOL copybooks, VSAM, AIX/AS400 extraction)
 * Financial services or insurance domain experience
 * Familiarity with multi-tier data certification frameworks (e.g., Bronze/Silver/Gold or Foundation/Production/Enterprise)
 * Experience with API gateways and AI integration patterns (e.g., Kong Gateway, APIGEE, MCP)
 * Experience designing and enforcing formal data contracts and dependency management
 * Consulting or client-facing experience with structured knowledge transfer
 * AWS certifications (Solutions Architect, Data Analytics) are a plus

## Description

* Build governed virtual data models in Denodo, including cross-system joins, canonical schemas, ABAC policies, and column-level masking
 * Configure and optimize semantic layer connectivity (JDBC/ODBC, connection pooling, failover)
 * Implement data quality rules (completeness, validity, uniqueness, anomaly detection, scoring)
 * Configure data catalog, lineage, and marketplace publishing
 * Build physical data products:
*

 + Source extraction (mainframe, CDC, COBOL copybooks)
 + ETL/ELT pipelines using Informatica IDMC or dbt
 + Data Lakehouse storage using Apache Iceberg on AWS S3
 * Define and implement machine-readable data contracts (YAML/JSON), including schema guarantees, SLAs, and dependency tracking
 * Develop APIs, SLA monitoring dashboards, reconciliation processes, and production-grade delivery patterns
 * Partner with client teams through demos, rotations, and structured knowledge transfer
 * Leverage AI-assisted development (Claude Code) to accelerate delivery and improve productivity


## Related Videos

- [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json)
- [CI/CD with Github Actions](https://www.wearedevelopers.com/videos/856-ci-cd-with-github-actions)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)