> Markdown version of [/jobs/ext/4349859-lead-big-data-administrator](https://www.wearedevelopers.com/jobs/ext/4349859-lead-big-data-administrator). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead, Big Data Administrator

- **Company:** Epsilon, Inc.
- **Location:** Irving, TX, United States (Remote available)
- **Experience:** Expert
- **Salary:** $138,715.0 - $182,000.0
- **Contract:** Temporary contract
- **Skills:** Software Applications, User Authentication, Bash Shell, Big Data, Computer-Aided Audit Tools, Information Engineering, Data Loss, Data Security, Disaster Recovery, Distributed Systems, Apache Hadoop, Hadoop Distributed File System, Apache Hive, Python (Programming Language), Kerberos (Protocol), Lightweight Directory Access Protocols (LDAP), Performance Tuning, Ansible, Prometheus, Shell Script, Apache Yarn, Grafana, Apache Spark, Apache Kafka, Data Pipelines
- **Published:** October 25, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=f61f51ccc6a14a6a

## About the Role

Requirements Employer will accept a bachelor's degree in Computer Applications or a related field followed by eight years of progressive, post-baccalaureate experience in the job offered or eight years of progressive, post-baccalaureate experience in any occupation in which the required experience was gained

## Description

Duties: Responsible for deploying and managing large-scale distributed systems, ensuring their performance, availability, and scalability; Oversee the installation, configuration, and maintenance of Hadoop ecosystems, including HDFS, Yarn, Hive, and Spark, as well as Kafka clusters; Implement and manage high-availability and disaster recovery strategies for both Hadoop and Kafka environments; Performance tuning, including resource allocation, cluster optimization, and monitoring, is a core responsibility to maintain system efficiency; Handle data pipeline orchestration and ensure seamless integration between Kafka producers, consumers, and downstream processing frameworks; Ensuring data security through encryption, access control, and auditing mechanisms; Troubleshoot and resolve complex issues related to cluster health, data loss, or Kafka offsets while maintaining minimal downtime; Automation of repetitive tasks using tools like Ansible, shell scripting in order to enhance the
operational efficiency; Collaborate closely with development, data engineering, and analytics teams to support evolving business requirements and optimize workflows; and Staying updated with emerging technologies in big data and streaming, which provide strategic guidance on system upgrades and architectural improvements., * Proficiency with the core Hadoop components and managing a cluster, including HDFS, Yarn, Hive, and Spark, as well as Kafka clusters.

 * Utilize ZooKeeper for coordination and synchronization within both Hadoop and Kafka clusters
 * Kerberos Security-Authentication: Kerberos, LDAP integration; Encryption: SSL/TLS, Hadoop KMS; Audit Tools: Ranger, Atlas, or custom solutions
 * Prometheus and Grafana to visualize metrics and set up alerts to track the health, performance, and resource utilization of clusters.
 * Diagnose and resolve complex issues related to cluster connectivity, performance degradation, and system failures
 * Fine-tune cluster configurations and optimize job performance by adjusting parameters related to memory, I/O, and replication
 * Bash and Python to automate routine tasks such as backups, log management, and cluster maintenance