> Markdown version of [/jobs/ext/1311320-lead-cloud-data-platform-engineer-ai-data-engineering](https://www.wearedevelopers.com/jobs/ext/1311320-lead-cloud-data-platform-engineer-ai-data-engineering). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Cloud Data Platform Engineer (AI & Data Engineering) - **Company:** THE JUDGE GROUP, INC. - **Location:** Chandler, AZ, United States - **Experience:** Expert - **Salary:** $143,520.0 - $153,920.0 - **Contract:** Permanent contract - **Skills:** Agile Methodology, Artificial Intelligence, Airflow, BigQuery, Cloud Computing, Cyber Security, Computer Programming, Data Architecture, Information Engineering, Data Governance, Dataspaces, Data Systems, Python (Programming Language), Meta-Data Management, Cloud Services, DataOps, Cloudera, Data Streaming, Enterprise Data Management, Data Processing, Google Cloud, Cloud Platform System, Data Ingestion, Apache Spark, Generative AI, HybridCloud, Pyspark, Apache Flink, Data Analytics, Real Time Data, Apache Kafka, Spark Streaming, Data Management, Data Lakehouse, Video Streaming, Cloud Migration, Virtual Agents, Stream Processing, Data Pipelines - **Published:** July 17, 2026 - **Apply:** https://www.dice.com/job-detail/73d17a9c-401b-412b-ab77-1cadbdbe1272 ## About the Role * Recent hands-on experience building AI-powered data solutions using: + LangChain + LangGraph or Agent Development Kit (ADK) + Agentic AI frameworks + Retrieval-Augmented Generation (RAG) + GraphRAG + Model Context Protocol (MCP) Data Engineering Experience * 5+ years of hands-on data engineering experience. * Experience designing and supporting cloud-based data platforms and processing frameworks. * Strong expertise building Spark-based ingestion and transformation solutions. Cloud Data Platform Experience * 3+ years of experience working with Data Lakehouse architectures and cloud-native data platforms. * Hands-on experience with: + Python + PySpark + Kafka + Apache Airflow + Google Cloud Storage (GCS) + BigQuery + Dataproc + Cloud Composer Streaming & Real-Time Data Processing * Experience developing and maintaining real-time data processing solutions utilizing: + Apache Kafka + Apache Flink + Spark Streaming Preferred Qualifications * Experience building AI-driven automation capabilities for enterprise data platforms. * Knowledge of cybersecurity data ecosystems and analytics environments. * Experience operating within Agile development teams. * Strong understanding of cloud migration strategies and hybrid cloud architectures. * Ability to influence technical direction and drive innovation across engineering organizations. Technical Skills Programming & Development * Python * PySpark Cloud Platforms * Google Cloud Platform (Google Cloud Platform) * Google Cloud Storage * BigQuery * Dataproc * Cloud Composer ## Description We are seeking a Lead Cloud Data Platform Engineer to build and modernize the Cyber Security Data Ecosystem on a hybrid cloud platform. This role combines advanced data engineering expertise with emerging AI technologies to develop scalable data products, automate data operations, and drive cloud transformation initiatives. The ideal candidate will have deep experience in cloud-native data platforms, real-time data processing, AI-powered data solutions, and modern Lakehouse architectures. You will play a key role in designing and implementing next-generation analytics capabilities while helping guide the migration from on-premises environments to cloud-based platforms. This position offers significant visibility and collaboration with principal engineers, product managers, architects, and data engineering teams across the organization., * Design, develop, and operationalize AI-enabled data platforms and data products on Google Cloud. * Build scalable data ingestion, transformation, and distribution pipelines supporting large-scale analytics and cybersecurity initiatives. * Utilize AI and agentic frameworks to automate data management, governance, quality monitoring, metadata management, and compliance processes. * Develop and maintain real-time and batch data processing solutions using modern streaming technologies. * Lead implementation of Lakehouse architectures and cloud-native data platforms. * Partner with engineers, architects, and business stakeholders to define technical roadmaps and prioritize strategic data initiatives. * Drive adoption of modern engineering standards, best practices, and emerging technologies across the data engineering organization. * Support cloud migration efforts from on-premises environments to Google Cloud-based architecture. * Mentor team members and provide technical leadership across multiple projects and initiatives. * Ensure solutions are secure, scalable, reliable, and aligned with enterprise data governance requirements., * Spark * Lakehouse Architecture * Data Pipelines * Data Modeling * Data Governance * Metadata Management Streaming Technologies * Apache Kafka * Apache Flink * Spark Streaming AI & Machine Learning * LangChain * LangGraph * Agent Development Kit (ADK) * Agentic Frameworks * RAG * GraphRAG * MCP Why Join This Opportunity? * Work on cutting-edge AI and data engineering initiatives within a large-scale enterprise environment. * Drive innovation in cloud modernization and data platform transformation. * Build intelligent data solutions that improve cybersecurity operations and analytics. * Collaborate with highly skilled engineers and technology leaders. * Gain exposure to advanced AI, cloud, and real-time data technologies at enterprise scale. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Coffee with Developers - Maria Apazoglou](https://www.wearedevelopers.com/videos/1209-coffee-with-developers-maria-apazoglou) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) ## Related Articles - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)