> Markdown version of [/jobs/ext/2365230-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2365230-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Abhyanth Solutions
- **Location:** Raritan, NJ, United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Unity 3d, Agile Methodology, Artificial Intelligence, Confluence, JIRA, Microsoft Azure, Cloud Engineering, Databases, Continuous Integration, Information Engineering, Data Governance, Data Integration, Data Transformation, Data Warehousing, IBM Cognos Business Intelligence, Python (Programming Language), PostgreSQL, Release Management, Power BI, SQL Databases, Tableau (Software), Teradata SQL, Enterprise Data Management, Azure Data Factory, Large Language Models, Apache Spark, Generative AI, Git, Data Lakes, Pyspark, Data Management, Databricks
- **Published:** August 28, 2026
- **Apply:** https://www.dice.com/job-detail/73d06940-01a2-48ba-8dd4-e9fdffa2911c

## About the Role

We are seeking an experienced Lead Data Engineer with 12+ years of experience in data engineering, data warehousing, and modern cloud-based data platforms. The ideal candidate will have strong hands-on expertise with Databricks, Apache Spark/PySpark, SQL, Python, Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF), along with experience working with enterprise data platforms and governance.

Required Technical Skills

 * Strong hands-on experience with Databricks, Apache Spark (PySpark), SQL, and Python.
 * Expertise in Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF).
 * Experience working with enterprise databases including Teradata and PostgreSQL.
 * Strong knowledge of Informatica and Data Warehousing concepts and technologies.
 * Familiarity with Power BI, Cognos, and Tableau is a plus.
 * Experience implementing and supporting Data Quality, Data Governance, and Data Reconciliation processes.
 * Proficiency with Jira, Confluence, Git, CI/CD, Release Management, and Agile Delivery methodologies.
 * Knowledge of Generative AI, LLMs, Databricks AI, and Natural Language Querying is preferred to support future AI-enabled procurement capabilities.
 * Knowledge of procurement and financial data domains is preferred.

Preferred Qualifications

 * Experience designing and delivering scalable cloud-based data engineering solutions.
 * Strong understanding of data integration, data transformation, data warehousing, and data governance.
 * Ability to work effectively in an Agile delivery environment and collaborate with cross-functional teams.
 * Experience supporting emerging AI/LLM capabilities within enterprise data platforms is highly desirable.
 * Procurement and financial data domain experience is a strong advantage.



## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)