Software Engineer - Kafka

This position provides technical guidance on complex strategic projects. Facilitates the design, development and maintenance of data streaming pipelines and distributed messaging system. Responsible for setting up Kafka clusters, configuring brokers, performance optimization and ensuring high availability. Recommends processes and products that meet strategic goals while aligning with the Bank’s long-term technology direction., * Infrastructure Management - Design and deploy Kafka clusters on premise to ensure scalability and high availability. Cluster management through capacity planning, monitoring and alerting.

• Pipeline Development - Develop and Maintain real-time, stream data pipes using Kafka connect, schema registry, streams, ksqldb.
• Performance Optimization - Monitor, tune and optimize brokers and topic configurations for throughput and low latency.
• Technical Expert - Maintains a strong knowledge of technology trends and impact on the Bank’s infrastructure to provide relevant consultative guidance. Trains and mentor peers on new technologies.
• Consultation - Provides technical advice and direction to associates, management, and developmental project teams. Utilizes expertise to consult and develop software to solve complex technical problems.
• Business Strategy - Acts as a trusted advisor on strategies that align developmental projects with long-term business needs and technological direction. Assists management in implementing initiatives or strategies across the division.
• Troubleshooting - Diagnose and work of remediations for issues related to infrastructure, consumer lags and message delivery.
• Security and Governance - Secure Kafka cluster via SSL and SASL, Manage data contracts using schema registry.

Bachelor’s Degree and 4 years of experience in Software application development and maintenance OR High School Diploma or GED and 8 years of experience in Software application development and maintenance

Preferred Area of Study: Computer Science, Information Systems Preferred Area of Experience: Information Technology, Application Development

Skill

• Strong understanding of Kafka architecture - topics, partitions, brokers, producers, consumers, connectors.
• Knowledge of distributed systems, data streaming and messaging systems.
• Experience with Confluent Kafka Connect, streams. Monitoring tools like, Grafana, Prometheus, Confluent Control Center.
• Ability to be on call for support on a rotating basis (as needed).