Hadoop Data Engineer

Cloudious LLC
Dallas, TX, United States
21 days ago
Apply on www.careerjet.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours

Tech stack

Java (Programming Language) Big Data Software Quality Continuous Integration Data Integration DevOps Distributed Data Store Distributed Systems Gradle Apache Hadoop MapReduce Apache Hive
+18 more
JD Edwards EnterpriseOne Apache Maven Performance Tuning SAP (Applications) Scala (Programming Language) SQL Databases Data Streaming Data Processing Sql Optimization Apache Spark Spring-boot Gitlab Apache Kafka Build Tools Spark Streaming Restful APIs Software Version Control Data Pipelines

Job description

The ideal candidate will be responsible for designing, building, and optimizing scalable data processing solutions, supporting both batch and real-time data pipelines. WHAT YOU’LL DO: Design, develop, and maintain scalable data processing applications using Java (8 or above) and Spring Boot. Build and optimize big data pipelines using Spark and Scala for large-scale data processing. Develop and consume RESTful web services for data integration and platform interoperability. Implement and manage batch and streaming data pipelines using Spark Streaming and Kafka. Write optimized Hive queries and SQL, focusing on performance and scalability. Work on distributed data platforms leveraging Hadoop ecosystem components. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Ensure code quality through reviews, version control, and CI/CD best practices. Troubleshoot and resolve performance, scalability, and data reliability issues

Requirements

Experience Required: 10+ years of hands-on experience in Java (version 8 or above). We are seeking an experienced Data Engineer with a strong background in Java, Spring Boot, Spark, and Scala, along with a deep understanding of big data and distributed systems., 8 12 years of hands-on experience in Java (version 8 or above). Strong expertise in Spring Boot 2 framework. Excellent proficiency in Spark. Strong experience with Hive, SQL optimization, Hadoop Map Reduce, and Kafka Messaging Bus. Solid understanding of REST Web Services. Experience with version control systems such as Version Management GitLab. Hands-on experience with build tools like Maven and/or Gradle. Knowledge of CI CD & DevOps is a plus Strong understanding of distributed systems and big data architecture. Experience with streaming frameworks such as Spark Streaming and Kafka Messaging Bus. Experience working in large-scale JD Edwards EnterpriseOne. Exposure to Performance Tuning and SAP PMO Capacity planning for big data systems

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.com
Prepare application

Good distractions

Loading talks and stories from around this role…