Cloud Data Engineer | Azure Data Platform + AI Integration

About the Opportunity A leading investment firm combining capital, innovation, and engineering is building the next generation of enterprise data infrastructure. We're looking for a hands-on Senior Cloud Data Engineer who lives at the intersection of Azure data engineering, AI, and DevOps automation and wants to turn raw data into intelligent, insight-driven automation at scale.

You'll own the full Microsoft data ecosystem (Azure Data Factory, Microsoft Fabric, Synapse Analytics, Logic Apps, Function Apps) while embedding Azure OpenAI and AI Services directly into production pipelines. You'll also bring AWS knowledge to support multi-cloud resilience and hybrid workloads.

This is a high-impact, high-visibility role where your pipelines and AI integrations will directly power smarter investment decisions across the firm.

What You'll Build & Own

• Design and deliver end-to-end, observable data pipelines and ingestion frameworks using Azure Data Factory and Microsoft Fabric with rock-solid data quality and governance
• Build serverless, event-driven automation with Azure Function Apps and Logic Apps, including orchestration that connects AI services to live data flows
• Own Infrastructure-as-Code end-to-end with Terraform across all environments
• Architect and run CI/CD for both data and AI workloads using GitHub Actions and Harness
• Embed Azure AI Services and Azure OpenAI into production workflows - document intelligence, embeddings, semantic search, prompt engineering, and AI-assisted data enrichment
• Design and manage analytical workloads in Azure Synapse Analytics (Lakehouse patterns, dedicated SQL pools, Spark transformations)
• Maintain hybrid/multi-cloud resilience by leveraging AWS services (S3, Glue, Lambda) for contingency pipelines
• Keep existing production pipelines healthy while continuously modernizing the platform

• ~10 years of data engineering experience with deep cloud-native expertise
• Expert-level hands-on skills with the full Microsoft Azure data stack: Azure Data Factory, Microsoft Fabric, Synapse Analytics, Logic Apps, and Function Apps
• Strong Terraform + CI/CD experience (GitHub Actions + Harness)
• Advanced SQL + Python (Python is critical for the AI/ML integration work we do)
• Proven track record integrating Azure OpenAI or Azure AI Services into real data pipelines (REST APIs, embeddings, prompt engineering, AI-assisted processing)
• Experience publishing data from ADF pipelines into SharePoint Lists using SharePoint Online REST APIs and OAuth
• Working knowledge of AWS (S3, Glue, Lambda) sufficient to design and operate contingency/hybrid workflows

• Direct impact on enterprise-wide analytics transformation and intelligent automation
• Work with cutting-edge Microsoft Fabric + Azure OpenAI in a production environment
• True ownership - you design it, build it, deploy it, and see it drive real business outcomes
• Collaborative, high-energy culture that values Integrity, Ability, and Energy
• Competitive compensation, strong benefits, and real professional growth in a firm that actually builds enduring technology, not just invests in it