Big Data Engineer with Java
Role details
Job location
Tech stack
Job description
In this role, you will design, develop, and maintain microservices using Java, Spark, Scala, Spring Boot and Python. You will be responsible for building and supporting scalable generic ingestion pipelines using Hive, NiFi, and HDFS on the Cloudera platform. Collaboration with ETL engineers will be key as you optimize snapshot-based ingestion and metadata-driven architectures. You will also develop and maintain monitoring dashboards using tools such as Kibana, while automating operational tasks through Bash and other scripting tools.
Your daily work will involve distributed systems on data platforms including Kafka, Spark, and Linux-based environments. You will integrate with DataStage and Oracle systems to support enterprise data workflows, and participate in code reviews, testing, and CI/CD processes to ensure high-quality delivery. Throughout, you will uphold best practices in software development, security, DevOps, and data engineering.
Requirements
Do you have experience in Spring Framework?, * Have strong Java programming skills with experience designing and developing microservices-based applications
- Have hands-on experience with Spring Boot, Apache Spark
- Have solid understanding of Big Data technologies, including Spark, Hive, HDFS, Kafka and NiFi
- Have working knowledge of databases, including Oracle/other RDBMS
- Feel comfortable working in Linux environments, with proficiency in shell scripting (Bash)
- Have hands-on experience with monitoring and visualization tools (e.g. Kibana)
- Have strong DevOps mindset, including experience with CI/CD pipelines.
- Have strong motivation to design and build scalable, reliable, and maintainable systems
- Have strong data-driven mindset with high digital fluency and a clear focus on data quality, reliability, and governance
- Have effective communication skills in English and the ability to collaborate confidently with diverse stakeholders
- Have proven ability to design structured, reusable processes and work in a disciplined, production-focused environment
- Are Team-oriented, proactive, and curious mindset with a continuous focus on learning and process improvement
- Have strong ownership and delivery mindset, covering the full lifecycle from development to production with a customer-centric approach
You'll get extra points for:
- Experience working with Azure DevOps and Kubernetes
- Experience with Google Cloud Platform (GCP) is a plus, enabling contribution to cloud-native and hybrid data initiatives
- Experience with Python, Scala and/or other JVM-based languages
- Familiarity with observabilityand monitoring tools such as Grafana, Prometheus, and the ELK stack
- Experience working in an Agile/Scrum development environment