Data Engineer - Full Stack Data Platform Support Engineer

• Spark
• Hadoop
• Scala / Python
• Amazon EMR
• Distributed Data Processing

• Large-Scale Batch & Streaming Workloads, We are seeking an experienced Data Engineer to support and maintain business-critical data platforms and cloud-native applications running on AWS. The ideal candidate will have strong experience with Big Data technologies, Spark, Hadoop, AWS, and distributed data processing systems. This role will focus on production support, platform reliability, troubleshooting, performance optimization, and automation of enterprise data platforms. The engineer will collaborate with SRE, Infrastructure, Development, and Business teams to ensure high availability and stability of batch and real-time data processing environments., Distinguished AI Engineer - Agentic AI Platform (Remote Eligible) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital…

• 17 hours ago +

• 6+ years of overall IT experience.
• 4+ years of experience in Data Engineering and Big Data technologies.
• Strong hands-on experience developing Spark applications using Scala/Python.
• Experience with distributed data processing and large-scale batch workloads.
• Experience building cloud-native applications on AWS and Kubernetes/EKS.
• Hands-on experience with Kafka-based streaming architectures.
• Experience designing ETL/data ingestion pipelines using Apache NiFi.
• Strong understanding of Hadoop ecosystem and data lake architecture.
• Experience working with relational and NoSQL databases.
• Experience with production support, troubleshooting, and performance optimization.
• Knowledge of CI/CD, Infrastructure as Code, and cloud security practices.

Other Qualifications

• Experience with microservices architecture and REST APIs.
• Experience with Kubernetes deployments and containerized applications.
• Knowledge of EMR cluster administration and Spark performance tuning.
• Experience with Terraform or CloudFormation.
• AWS Certification (Associate or Professional) preferred.
• Strong analytical, debugging, and communication skills.