> Markdown version of [/jobs/ext/4720910-it-technology-lead-architecture-togaf](https://www.wearedevelopers.com/jobs/ext/4720910-it-technology-lead-architecture-togaf). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# IT - Technology Lead | Architecture | TOGAF

- **Company:** Ipolarity LLC
- **Location:** Raleigh, NC, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Query Performance, Artificial Intelligence, Airflow, Amazon Web Services, ARM Architecture, Automation of Tests, Unit Testing, Microsoft Azure, Big Data, Cloud Computing, Code Generation, Software Quality, Code Review, Cyber Security, Continuous Integration, Data Discovery, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, DevOps, Data Intelligence, Python (Programming Language), Performance Tuning, Systems Development Life Cycle, Software Tools, Standard Sql, DataOps, Requirements Management, Software Engineering, SQL Databases, Enterprise Data Management, Azure Data Factory, GitHub Copilot, Retrieval-Augmented Generation, System Availability, Large Language Models, Snowflake, Prompt Engineering, Mttr, Technical Debt, Generative AI, Togaf, Pyspark, Kubernetes, Deployment Automation, Data Management, Machine Learning Operations, Virtual Agents, Data Pipelines, Databricks
- **Published:** November 14, 2026
- **Apply:** https://www.careerjet.com/jobad/us930eff838b8d93f541a64bfab0fc4db1

## About the Role

10+ years of experience in Data Engineering, Data Platforms, Data Warehousing or related technology roles.
Strong hands-on experience with enterprise-scale Data Engineering and Data Warehouse platforms.
Strong expertise in Snowflake and/or Databricks.
Advanced knowledge of SQL, Python and/or PySpark.
Experience designing and optimizing ETL/ELT pipelines and large-scale data-processing solutions.
Experience with AI-assisted engineering tools such as:
o GitHub Copilot
o Snowflake Cortex
o Databricks Genie
o Other enterprise Generative AI/code-assistance capabilities
Strong understanding of Snowflake performance optimization, workload management and cost optimization.
Experience with CI/CD, DevOps, automated testing and modern Data Engineering SDLC practices.
Experience implementing data observability, monitoring, anomaly detection or automated remediation.
Understanding of Generative AI, LLMs, prompt engineering and enterprise AI governance.
Ability to translate AI capabilities into practical engineering solutions and measurable business outcomes.
Strong stakeholder management and communication skills, with the ability to influence engineering, architecture and leadership teams.
Preferred Skills
Experience working with Azure/AWS enterprise data platforms.
Experience with Azure Data Factory, dbt, Airflow or similar orchestration technologies.
Knowledge of data observability and data-quality frameworks.
Experience building reusable Data Engineering accelerators or frameworks.
Understanding of MLOps/LLMOps, RAG and AI agent/agentic workflow concepts.
Experience implementing enterprise AI governance and secure Generative AI adoption.
Exposure to DataOps, DevOps, FinOps and Platform Engineering practices.
Minimum Years of Experience:
8 to 10+ years
Certifications Needed

## Description

We are looking for an experienced AI & Intelligent Automation Lead Data Engineering to drive the adoption of AI-assisted engineering and intelligent automation across our enterprise data platform.
The role will be responsible for defining and executing an AI-first Data Engineering strategy focused on accelerating development, improving data quality and platform reliability, automating operational activities, reducing technical debt, and optimizing cloud and data-platform costs.
The successful candidate will combine strong Data Engineering and Data Platform architecture expertise with practical experience applying AI capabilities such as GitHub Copilot, Snowflake Cortex and Databricks Genie across the Software Development Lifecycle., AI & Intelligent Automation Strategy
Define and execute the AI-enabled Data Engineering strategy to improve engineering velocity, quality, reliability, scalability and operational efficiency.
Identify, assess and prioritize high-value AI and automation use cases across Data Engineering, Data Warehouse and BI teams.
Develop reusable patterns and accelerators for introducing AI into existing Data Engineering and platform operations.
Build an AI-first engineering culture by adopting driving, enablement, governance and measurable engineering outcomes.
AI-Assisted Data Engineering
Lead adoption of GitHub Copilot, Snowflake Cortex and Databricks Genie to accelerate engineering activities and improve developer productivity.
Implement AI-assisted approaches for generating SQL, Python, PySpark, pipeline code, unit tests, documentation and deployment artifacts.
Automate generation of standardized, reusable data pipelines and engineering components.
Target automation of approximately 30 50% of repetitive pipeline development activities, where technically and operationally appropriate.
Build reusable engineering templates, prompts, coding standards and AI-assisted development patterns.
Intelligent Data Operations & Self-Healing Pipelines
Implement AI-driven monitoring and anomaly detection to proactively identify:
o Data quality issues
o Pipeline failures
o Schema changes and drift
o Processing anomalies
o Performance degradation
Design self-healing data pipelines capable of detecting and automatically resolving known operational issues.
Apply intelligent automation to incident triage, troubleshooting and root-cause analysis.
Reduce manual L1/L2 operational effort through automated diagnosis, remediation and preventive monitoring.
Establish appropriate human-in-the-loop controls for scenarios where automated remediation could create downstream risk.
Snowflake & Databricks Optimization
Leverage Snowflake AI and Cortex capabilities to improve engineering productivity and platform intelligence.
Identify expensive or inefficient queries and workloads and implement appropriate optimization strategies.
Improve warehouse utilization, workload management, query performance and overall platform cost efficiency.
Utilize Databricks Genie and Databricks AI capabilities to improve data discovery, analytics accessibility and engineering productivity.
Work with platform teams to establish measurable FinOps outcomes around compute consumption and cloud cost optimization.
AI-Enabled SDLC
Champion the use of AI across the complete engineering lifecycle, including:
Requirements analysis and refinement
Solution and pipeline design
Code generation and refactoring
Unit and integration testing
SQL and PySpark optimization
Code reviews
Technical documentation
CI/CD and deployment automation
Production monitoring
Incident analysis
Root-cause analysis
Technical debt identification and remediation
Ensure AI-generated outputs are appropriately reviewed, tested and integrated into existing engineering standards.
Responsible AI & Governance
Define standards and guardrails for responsible use of Generative AI within Data Engineering.
Establish controls covering:
o Data privacy
o Information security
o Regulatory compliance
o Intellectual property
o Code quality
o AI-generated code validation
o Accuracy and hallucination risk
o Maintainability and auditability
Ensure sensitive enterprise information is protected when engineers use AI-assisted development tools.
Work with architecture, security, risk and governance teams to establish approved AI engineering patterns.
Metrics & Value Realization
Define and maintain measurable KPIs for AI adoption and engineering effectiveness, such as:
AI tool adoption across engineering teams
Developer productivity improvements
Percentage of repetitive activities automated
Development cycle-time reduction
Pipeline failure rate
Mean Time to Detect (MTTD)
Mean Time to Resolve (MTTR)
Data-quality incident reduction
Technical-debt reduction
Cloud and platform cost savings
Reuse of standardized engineering components
The role will be expected to demonstrate measurable business value from AI adoption rather than focusing only on tool deployment., Top 3 responsibilities you would expect the Subcon to shoulder and execute:
Work closely with client and try to understand the requirements
Manage offshore and onsite team
Take the responsibility
Interview Process (Is face to face required?)
Virtual
Any additional information you would like to share about the project specs/nature of work:
Project Code: Arch Re 2023 DNA HYD1STP