> Markdown version of [/jobs/ext/2746479-it-technology-lead-big-data-hadoop-hadoop](https://www.wearedevelopers.com/jobs/ext/2746479-it-technology-lead-big-data-hadoop-hadoop). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# IT - Technology Lead | Big Data - Hadoop | Hadoop

- **Company:** Cloudspace LLC
- **Location:** St. Louis, MO, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Airflow, Amazon Web Services, Amazon Elastic Compute Cloud, Amazon S3, Data Analysis, Audit Trail, Big Data, Cloud Computing, Continuous Integration, Data Infrastructure, Linux, DevOps, Github, Apache Hadoop, Identity and Access Management, Python (Programming Language), Performance Tuning, Red Hat Enterprise Linux, Ansible, Prometheus, Cloudera, Shell Script, Data Streaming, Transport Layer Security, Cloud Platform System, System Availability, Grafana, Infrastructure as Code (IaC), Amazon Virtual Private Cloud (VPC), Git, Cloudformation, Containerization, Gitlab-ci, Kubernetes, Infrastructure Automation Frameworks, Information Technology, Apache Kafka, CruiseControl, Cloudwatch, Terraform, Splunk, Software Version Control, Docker, Jenkins
- **Published:** September 6, 2026
- **Apply:** https://www.careerjet.com/jobad/usd469418c2bc0a5dc6f2cb148f8a3e491

## About the Role

Docker and Kubernetes.Develop automation scripts using Python, Shell, or Bash.Operations & SupportProvide Level 2 and Level 3 production support.Participate in on-call support rotations and incident management activities.Perform root cause analysis (RCA) and implement preventive measures.Create and maintain operational documentation and standard operating procedures.Ensure compliance with security, audit, and regulatory requirements.Required SkillsApache Kafka AdministrationKRAFT and MSKAWS Cloud ServicesLinux (RHEL/Rocky Linux)Shell Scripting and PythonJenkins, Git, Ansible, TerraformDocker and KubernetesMonitoring Tools (Grafana, Prometheus, Splunk)Networking, Security, and High Availability ConceptsPerformance Tuning and Capacity PlanningPreferred Qualifications:Bachelor s degree in Computer Science, Information Technology, or related field.Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.AWS, GCP, Kafka, or Kubernetes certifications.Experience supporting large-scale
production environments handling petabyte-scale data workloads.Key Achievements ExpectedMaintain platform availability above 99.9%.Adopt AI-assisted engineering practices to improve operational efficiency, reduce manual effort, and accelerate troubleshooting and documentation.Automate repetitive operational tasks.Improve cluster performance and resource utilization.Ensure secure, scalable, and reliable data platform operations.Support enterprise data engineering, analytics, and AI/ML workloads efficiently.Minimum years of experience8-10 yearsCertifications Needed :NoTop 3 responsibilities you would expect the Subcon to shoulder and executePerform infrastructure provisioning and automation using Infrastructure as Code IaC.Design and maintain CICD pipelines using Jenkins, GitHub Actions, GitLab CI, or similar tools. Automate infrastructure deployment using Terraform, CloudFormation, and AnsibleManage source control repositories and release processes. Implement monitoring and alerting

## Description

Job Title: Kafka AdminWork Location: Saint LouisMO63131Vendor Rate: 68.40Contract duration: 12Target Start Date: 31 Aug 2026Does this position require Visa independent candidates only? YesInterview Process (Is face to face required?) Yes, face to face is mandatory for Interview**onsite work**Job Details:Detailed Job DescriptionExperienced Big Data Administrator responsible for managing, supporting, automating, and optimizing enterprise-scale Hadoop, Kafka, AWS cloud, and DevOps platforms. The role focuses on ensuring high availability, performance, security, scalability, and operational excellence across big data ecosystems while collaborating with development, architecture, and business teams.Key Responsibilities:Kafka AdministrationDeploy, configure, and manage Apache Kafka clusters and AWS MSK environments.Monitor broker health, partitions, replication factors, and consumer lag.Perform capacity planning and cluster scaling activities.Manage Kafka security using SSL, SASL, ACLs, and
encryption standards.Troubleshoot producer, consumer, and broker performance issues.Support Kafka Connect, Schema Registry, Cruise Control, and MirrorMaker implementations.AWS Cloud AdministrationManage cloud infrastructure services including EC2, S3, IAM, VPC, EBS, CloudWatch, CloudTrail and AWS Glue.Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow environments.Implement cloud security best practices and governance controls.Perform infrastructure provisioning and automation using Infrastructure as Code (IaC).Monitor cloud resource utilization and optimize operational costs.DevOps & AutomationDesign and maintain CI/CD pipelines using Jenkins, GitHub Actions, GitLab CI, or similar tools.Automate infrastructure deployment using Terraform, CloudFormation, and Ansible.Manage source control repositories and release processes.Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk, ELK, or CloudWatch.Support containerization technologies such as


## Related Videos

- [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [Industrializing your Data Science capabilities](https://www.wearedevelopers.com/videos/178-industrializing-your-data-science-capabilities)
- [Docker exec without Docker](https://www.wearedevelopers.com/videos/1094-docker-exec-without-docker)
- [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker)

## Related Articles

- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Best US AI Conferences for CTOs in 2026: Build vs. Buy, Vendor Evaluation, and Peer Intelligence](https://www.wearedevelopers.com/magazine/736-best-us-ai-conferences-for-ctos-in-2026-build-vs-buy-vendor-evaluation-and-peer-intelligence)
- [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)