Lead Cloud Engineer (Software Engineer

Platform & Architecture Leadership

• Lead design and operations of OpenShift clusters or Google Cloud Platform projects running containerized Spark workloads
• Architect enhancements for the statistical model execution platform
• Drive cloud modernization, automation, and infrastructure scalability initiatives

Distributed Data Processing

• Architect and optimize Apache Spark data pipelines at enterprise scale
• Improve performance, reliability, and cost-efficiency using Kubernetes orchestration
• Integrate Spark with Kafka, S3S, data lakes, and enterprise databases

Application & Tooling Development

• Lead development of Python microservices and backend applications (Django preferred)
• Build tools that support statisticians, data scientists, and model developers
• Ensure high reliability, scalability, and user-focused design

Automation & CI/CD

• Design and govern CI/CD systems using GitHub Actions, Helm, Sonar, Harness, etc.
• Automate deployment, environment provisioning, monitoring, and testing

Monitoring & Reliability

• Manage and improve cluster health using Prometheus, Grafana, and Google Cloud Platform observability tools
• Troubleshoot distributed systems and optimize resource allocations

Security & Compliance

• Enforce enterprise security standards and regulatory requirements
• Implement RBAC, encryption, and secure development practices

Cross-Functional Leadership

• Collaborate with data scientists, DevOps, architecture, and platform teams
• Mentor engineers across multiple experience levels
• Lead production releases, incident response, and root-cause analyses

Experience with large-scale data systems, distributed computing, or financial-services environments is valuable. Skills in AI/ML tooling or platform enablement are a strong plus., * 5-7+ years using Apache Spark for large-scale data processing

• 3+ years developing Python/Django backend applications
• 3+ years building platforms or tools for high-volume data users
• 3-5+ years working with Google Cloud (or similar major cloud provider)
• 3+ years operating Kubernetes / OpenShift containerized workloads
• Experience in financial-services, risk, or regulated environments preferred

Technical Skills

• Strong Spark (PySpark, Scala, or Java) expertise
• Deep understanding of Docker and Kubernetes architecture
• Hands-on OpenShift administration
• Experience with conda Python environments
• Strong coding skills in Python, Scala, or Java
• CI/CD experience (GitHub Actions, Helm, Sonar, Harness)
• Proficient in debugging distributed systems end-to-end
• Knowledge of Google Cloud Platform (GCS, IAM, GKE, Cloud Run, etc.)

Education

• Bachelor's degree in Computer Science, Engineering, or related field

Role Level

• Lead Engineer: 8-10+ years with demonstrated experience guiding complex technical platforms