Data Engineer - INTL India

Insight Global
Pittsburgh, United States of America
4 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 32K

Job location

Pittsburgh, United States of America

Tech stack

Artificial Intelligence
Airflow
Azure
Software Quality
Computer Programming
Information Engineering
Data Infrastructure
Data Integrity
ETL
IBM DB2
Data Intelligence
Python
Metadata
Microsoft SQL Server
SQL Azure
Oracle Applications
SQL Databases
Systems Integration
Teradata
Cloud Platform System
Azure
Sql Optimization
Large Language Models
Spark
Generative AI
Data Lake
AI Platforms
Data Pipelines
Databricks
Control M

Job description

We are seeking a Senior Data Engineer with AI/GenAI expertise to join the Data Platform (Foundry) team. This role will focus on building and optimizing enterprise-scale data pipelines while also enabling metadata-driven, AI-driven use cases, including copilots, LLM integrations, and intelligent data workflows using Databricks, Azure, ADF, Azure SQL, Astronomer/Airflow or Control M., Design, build, and maintain data pipelines using Databricks, Spark, ADF, Advanced SQL/Python, and Azure services Develop and optimize bronze * silver * gold transformations Schedule and orchestrate via ADF, Airflow, Control M. Airflow experience would be a huge plus Enable AI/GenAI use cases across the platform, including: Copilot integrations LLM-based workflows Intelligent data enrichment pipelines Work with Databricks GenAI capabilities . Integrate external AI platforms (e.g., Anthropic, Azure OpenAI) into enterprise data workflows Ensure data reliability, performance, and operational stability Troubleshoot and resolve production issues with strong SLAs Support ingestion using tools such as: Fivetran / ADF / Auto Loader Implement data quality, monitoring, and reconciliation frameworks Ability to read/write code in SQL, Python, other Collaborate across: Data Engineering teams Data Platform engineering Business stakeholders Contribute to platform modernization and AI adoption strategy Follow governance best practices: Code quality Deployment controls Audit and compliance

Requirements

8-10+ years in Data Engineering Strong experience with: Databricks (Delta Lake, Spark) Azure Data Platform - ADF, Synapse SQL - Azure SQL, SQL Server, Teradata, DB2, Oracle Hands-on experience with AI/GenAI technologies, including: Building or integrating copilot-style experiences Working with LLMs (e.g., Anthropic Claude, OpenAI models) Strong programming skills: Python, SQL Orchestration using - Airflow or Astronomer, Control M. Preferably Airflow Experience building scalable ETL/ELT pipelines Familiarity with production support and incident management

Benefits & conditions

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.

Apply for this position