Lead Software Engineer - AI & Cloud Data Engineering

We're building a modern, AI-enabled data platform that supports large-scale analytics and automation in a complex, hybrid-cloud environment. As a Lead Software Engineer - AI & Cloud Data Engineering, you will play a critical role in designing, building, and operationalizing cloud-native data and AI solutions that support next-generation analytics and security use cases.

You will work in a fast-paced, highly collaborative environment alongside principal engineers, product managers, and data engineers to deliver scalable data pipelines and AI-driven automation capabilities. This is a visible, hands-on technical leadership role with meaningful impact across a large data ecosystem., In this role, you will:

• Lead the design and implementation of cloud-native, AI-enabled data pipelines on Google Cloud to ingest, transform, and distribute large-scale data.
• Build and operate streaming and batch data processing solutions using technologies such as Kafka, Spark, and Flink.
• Apply AI and agentic frameworks (e.g., LangChain, LangGraph, RAG/GraphRAG) to automate data management, governance, quality, metadata, and compliance workflows.
• Consult on complex, large-scale data engineering initiatives, contributing to architectural decisions and long-term platform strategy.
• Collaborate strategically within a matrixed organization, partnering with engineers, product managers, and stakeholders to roadmap and deliver high-priority data capabilities.
• Drive adoption of modern data and AI technologies through technical leadership, mentorship, and hands-on development.
• Support the migration of on-premises data platforms to Google Cloud-based architectures.
• Ensure solutions align with organizational standards, security requirements, and compliance expectations., * Opportunity to build a modern AI-powered data platform from the ground up
• High-impact work with visibility across engineering and product teams
• Exposure to cutting-edge AI, data engineering, and cloud-native technologies
• Collaborative environment that values innovation, technical excellence, and continuous learning

By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.

• 5+ years of experience in software or data engineering, or equivalent experience gained through work, consulting, education, military service, or training.
• Experience working in large-scale data environments.
• Hands-on experience building and supporting cloud data solutions, including Spark-based ingestion and processing.
• Strong proficiency in Python and experience with Kafka.
• Demonstrated experience developing data pipelines on Google Cloud, including services such as:

• Google Cloud Storage
• BigQuery
• Dataproc
• Cloud Composer (Airflow)

• Experience with data lakehouse architectures and modern data processing patterns.
• Proven ability to work on complex, ambiguous technical problems requiring in-depth analysis and sound judgment.

Preferred Qualifications

• Recent, hands-on experience with AI tooling and frameworks, including:

• LangChain
• LangGraph / ADK
• Agentic frameworks
• Retrieval-Augmented Generation (RAG) and GraphRAG
• MCP-based agent architectures

• Experience using AI to accelerate data engineering, including code generation, context engineering, and prompt engineering.
• Hands-on experience with streaming technologies (Kafka, Flink, Spark Streaming).
• Strong background in cloud-based data lakes and warehouses with automated data pipelines.
• Public cloud certifications (e.g., Google Cloud Platform Professional Data Engineer, Azure Data Engineer, AWS Data Analytics Specialty).
• Experience with web-based UI development (React, Node.js) a plus.