Lead Data Engineer

Abhyanth Solutions

Raritan, NJ, United States

8 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Unity 3d Agile Methodology Artificial Intelligence Confluence JIRA Microsoft Azure Cloud Engineering Databases Continuous Integration Information Engineering Data Governance Data Integration

+20 more

Data Transformation Data Warehousing IBM Cognos Business Intelligence Python (Programming Language) PostgreSQL Release Management Power BI SQL Databases Tableau (Software) Teradata SQL Enterprise Data Management Azure Data Factory Large Language Models Apache Spark Generative AI Git Data Lakes Pyspark Data Management Databricks

Requirements

We are seeking an experienced Lead Data Engineer with 12+ years of experience in data engineering, data warehousing, and modern cloud-based data platforms. The ideal candidate will have strong hands-on expertise with Databricks, Apache Spark/PySpark, SQL, Python, Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF), along with experience working with enterprise data platforms and governance.

Required Technical Skills

Strong hands-on experience with Databricks, Apache Spark (PySpark), SQL, and Python.
Expertise in Microsoft Azure, Delta Lake, Unity Catalog, and Azure Data Factory (ADF).
Experience working with enterprise databases including Teradata and PostgreSQL.
Strong knowledge of Informatica and Data Warehousing concepts and technologies.
Familiarity with Power BI, Cognos, and Tableau is a plus.
Experience implementing and supporting Data Quality, Data Governance, and Data Reconciliation processes.
Proficiency with Jira, Confluence, Git, CI/CD, Release Management, and Agile Delivery methodologies.
Knowledge of Generative AI, LLMs, Databricks AI, and Natural Language Querying is preferred to support future AI-enabled procurement capabilities.
Knowledge of procurement and financial data domains is preferred.

Preferred Qualifications

Experience designing and delivering scalable cloud-based data engineering solutions.
Strong understanding of data integration, data transformation, data warehousing, and data governance.
Ability to work effectively in an Agile delivery environment and collaborate with cross-functional teams.
Experience supporting emerging AI/LLM capabilities within enterprise data platforms is highly desirable.
Procurement and financial data domain experience is a strong advantage.