Lead Cloud Data Platform Engineer
Role details
Job location
Tech stack
Job description
In this contingent resource assignment, you will consult on complex initiatives with broad impact and large-scale planning for Software Engineering. As a Lead Cloud Data Platform Engineer, you will help build next-generation data products and AI-enabled automation capabilities supporting the Cyber Security Data Ecosystem. This role is focused on modern cloud-native data architecture, agentic AI frameworks, large-scale data processing, and analytics modernization as continues its cloud transformation journey.
The successful candidate will work closely with engineers, architects, product managers, and cybersecurity stakeholders to design and implement scalable data solutions that leverage AI to improve data governance, automation, quality, and consumption. Day-to-Day Responsibilities
- Design, develop, and operationalize cloud-native data platforms on Google Cloud
- Build scalable data ingestion, transformation, and distribution pipelines using Spark-based technologies
- Develop AI-enabled data products and automation capabilities
- Leverage Agentic AI frameworks including:
- LangChain
- LangGraph
- ADK
- MCP
- RAG
- GraphRAG
- Implement AI-driven solutions for:
- Data Governance
- Data Quality
- Metadata Management
- Data Compliance
- Data Discovery
- Develop real-time and batch data processing capabilities using:
- Kafka
- Spark Streaming
- Flink
- PySpark
- Design and support Data Lakehouse architecture and cloud analytics platforms
- Collaborate with principal engineers, product managers, and data engineering teams to define and deliver strategic data capabilities
- Support cloud migration initiatives and modernization efforts
- Promote engineering best practices and mentor team members on modern cloud and AI technologies
Requirements
-
Strong experience with Cloud Data Engineering on Google Cloud Platform (Google Cloud Platform) including BigQuery, DataProc, Cloud Storage, and Cloud Composer
-
Hands-on experience with AI/Agentic frameworks including LangChain, LangGraph, ADK, RAG, GraphRAG, MCP, and agent-based architectures
-
Experience building and supporting large-scale data pipelines using Python, PySpark, Kafka, Spark Streaming, Flink, and AirflowPlusses
-
Experience developing AI-powered data products and automation frameworks
-
Experience with Data Lakehouse architecture and large-scale analytics platforms
-
Cybersecurity data or security analytics experience
-
Experience leading cloud migration initiatives