Kubernetes Big Data Engineer

This is Hima from Appridat Solutions LLC. I was reviewing your resume online and would like to talk to you regarding an exciting opportunity for Senior Kubernetes Big Data Engineer (Kubernetes / AWS / Spark) at Rockville, MD or McLean, VA. We have with one of Appridat Solutions LLC premier clients., We are seeking an experienced Senior Big Data Engineer to help modernize and evolve a large-scale enterprise data platform supporting mission-critical analytics and data processing. This role is ideal for an engineer who enjoys solving complex data challenges, building cloud-native solutions, and working with massive datasets in distributed environments.

You will play a key role in modernizing existing big data infrastructure by migrating traditional data processing workloads to containerized, Kubernetes-based platforms while partnering with engineers across architecture, platform engineering, and data science. This position offers the opportunity to work with modern cloud technologies, AI-assisted development tools, and next-generation data engineering practices.

What You’ll Do

Design, develop, and optimize highly scalable big data pipelines using Spark, Python, Scala, SQL, and distributed processing frameworks.

Modernize existing big data workloads by migrating EMR and EC2-based solutions to Kubernetes and cloud-native containerized architectures.

Build, deploy, and support Spark applications running on Kubernetes and Amazon EKS.

Engineer high-performance data solutions capable of processing terabyte- and petabyte-scale datasets.

Optimize data pipelines, Spark jobs, and distributed workloads for performance, scalability, resiliency, and cost efficiency.

Design data ingestion, transformation, and storage solutions that support enterprise analytics and reporting.

Troubleshoot production data pipelines, Kubernetes environments, and distributed processing issues.

Collaborate with architects, software engineers, analysts, and business stakeholders to translate requirements into scalable technical solutions.

Develop automated testing, CI/CD processes, and deployment pipelines to improve software quality and delivery.

Contribute to technical documentation, engineering standards, and solution design throughout the software development lifecycle.

Evaluate and leverage emerging technologies, including AI-assisted development tools, to improve engineering productivity.

5+ years of experience building enterprise-scale Big Data solutions.

Strong hands-on experience with:

Kubernetes

Apache Spark

AWS cloud services

Python and/or Scala

Advanced SQL

Experience processing and optimizing large-scale distributed data workloads (terabytes or greater).

Strong understanding of Spark architecture, performance tuning, partitioning, resource management, and distributed computing concepts.

Experience designing, deploying, and supporting containerized applications in Kubernetes.

Experience with CI/CD pipelines using tools such as GitLab, Bitbucket, Jenkins, or similar platforms.

Strong problem-solving skills with the ability to optimize algorithms, troubleshoot complex production issues, and improve system performance.

Excellent communication skills with experience collaborating across technical and business teams.

Preferred Qualifications

Experience with Elasticsearch and distributed search technologies.

Experience with Hadoop, Hive, Trino, or similar big data ecosystems.

Experience with Amazon EKS, EMR on EKS, Glue, Athena, Lambda, or other AWS analytics services.

Familiarity with Infrastructure as Code (Terraform or CloudFormation).

Experience with monitoring and observability tools such as Prometheus, Grafana, ELK, or CloudWatch.

AWS, Kubernetes (CKA/CKAD), or Big Data certifications.

Financial services or other highly regulated industry experience.

Modern Engineering Environment

We’re looking for engineers who embrace modern software development practices and continuously evaluate new technologies to improve delivery. Experience using AI-assisted engineering tools such as GitHub Copilot, ChatGPT, Claude, Amazon Kiro, Model Context Protocol (MCP), or agentic AI workflows is highly valued.

If you’re passionate about cloud-native architecture, Kubernetes, large-scale data engineering, and building the next generation of enterprise data platforms, we’d love to hear from you.