Data Engineer

• Responsible for developing software product which requires Spark on K8s with scala & Python programming skills.
• Responsible for building test automation suite
• Refactoring of existing code base to the high standards and detect hotspots.
• Responsible for enhancing the code base and meet the code coverage of 80%
• Responsible for supporting end customer use cases, identify gaps, bugs and new features required to fulfil end customer requirement.
• Responsible for building automation utilities to reduce redundant work., * Designs and writes complex code in several languages relevant to our existing product stack, with a focus on automation
• Configures, tunes, maintains and installs applications systems and validates system functionality
• Installs new software releases and application system upgrades. Evaluates and installs software patches
• Monitors and fine tunes applications system to achieve optimum performance levels and works with hardware teams to resolve issues with hardware and software
• Assists with application system problem resolution by working with application developers, vendors, and internal infrastructure teams member to troubleshoot
• Addresses product backlog and provide continuous delivery of high-quality features
• Maintains a comprehensive operating system hardware and software configuration database/library of all supporting documentation to ensure data integrity
• Acts to improve the overall reliability of systems and to increase efficiency
• Works collaboratively with cross functional teams, using Agile / DevOps principles to bring products to life, achieve business objectives and serve customer needs

• Kubernetes Very Strong and #1 4 to 5 years
• Data Pipelines ETL Preferred Bring data to send back to other team
• Understanding of Python is good and will code in Python Not working on APIs
• Good Understanding of Machine Learning Pipelines
• Argo WorkFlow Experience
• Docker and Jenkins Would be good
• Workflow Experience Would be good, * Spark on K8s
• Cloud knowledge
• Spark with scala/Java
• Python to support/contribute to Airflow orchestration.
• ETL knowledge
• Spark structured streaming
• Knowledge on Kafka
• Knowledge on Splunk

Desired Skills:

• Apache Flink for real-time streaming.
• Azure knowledge
• Springboot microservice knowledge