Lead Scala Data Engineer

We are seeking an experienced Lead Scala Data Engineer to serve as the primary technical owner of an enterprise data-processing framework supporting Medicaid Encounter Processing and enterprise data ingestion. \n

\n This is a hands-on technical ownership position responsible for maintaining and enhancing a mission-critical production application built with Scala, Apache Spark, Hive, Drools, and Cloudera Data Platform (CDP). The ideal candidate will have extensive experience developing distributed data-processing applications and supporting them in production. \n

\n Key Responsibilities \n \n

• Serve as the primary technical owner of the Scala/Spark application framework supporting Medicaid Encounter Processing.\n
• Maintain and enhance production applications developed using Scala, Spark, Hive, and Drools.\n
• Own Drools business-rule implementation and ongoing rule maintenance.\n
• Support enterprise ingestion and processing of provider, member, reference, eligibility, and encounter data.\n
• Own Spark and Hive batch-processing workflows.\n
• Troubleshoot production issues, identify root causes, and resolve application defects.\n
• Implement business, regulatory, and application changes.\n
• Manage production releases, version control, and deployment coordination.\n
• Perform Spark performance tuning and optimization.\n
• Monitor and provide basic operational support for the Cloudera Data Platform (CDP).\n
• Maintain technical documentation, operational procedures, and knowledge-transfer materials.\n
• Coordinate with infrastructure, cloud operations, QA, business, and other technical teams.\n

• 7+ years of experience developing enterprise-scale distributed data-processing applications.\n
• Strong hands-on Scala development experience, preferably 4-6+ years.\n
• 4-6+ years of Apache Spark and Hive development experience.\n
• Significant experience developing and maintaining applications on Cloudera Data Platform (CDP) 7.x or equivalent enterprise Hadoop environments.\n
• Hands-on experience implementing business rules using the Drools Rules Engine, preferably 4-6 years.\n
• Strong SQL development and query-optimization skills.\n
• Experience supporting Linux-based production environments.\n
• Strong experience troubleshooting distributed Spark applications in production.\n
• Experience with Git and modern version-control practices.\n
• Demonstrated experience taking technical ownership of production applications, including incidents, defects, enhancements, releases, deployments, and performance issues.\n, * Medicaid or healthcare industry experience.\n
• Experience with Medicaid Encounter Processing.\n
• Experience with Cloudera Manager, HDFS, and YARN.\n
• Experience integrating with IBM DataStage.\n
• Familiarity with AWS infrastructure supporting Cloudera.\n
• Experience working in Agile environments using Jira and Confluence.\n, n We are specifically looking for a Scala/Spark Data Engineer with strong Cloudera/CDP experience, rather than a Cloudera Administrator. The successful candidate should be capable of independently owning a production application across the complete Scala + Spark + Hive + Drools + Cloudera technology stack. \n