ETL Developer (Mid - SME Level) - Top Secret with FS Poly
Role details
Job location
Tech stack
Job description
Xcelerate Solutions is seeking an experienced ETL Developer to play a pivotal role in shaping, leading, and implementing cutting-edge data flow solutions centered around Apache NiFi. This role supports the design, development, and deployment of Extract, Transform, and Load (ETL) processes that power enterprise-level data lakes and analytics ecosystems. The ideal candidate will define architectural best practices, optimize performance in large-scale environments, and mentor junior developers, ensuring the delivery of robust, scalable, and secure data flow solutions aligned with mission-critical customer objectives., * Architect complex data pipelines: Design and implement enterprise-level NiFi-based ETL architectures for large-scale data ingestion, transformation, and processing from diverse structured and unstructured sources.
- Optimize and tune performance: Configure and fine-tune NiFi processors, manage system memory, and balance workloads to ensure optimal throughput and efficiency for both batch and real-time data flows.
- Advanced troubleshooting: Identify and resolve complex data flow issues, performance bottlenecks, and integration challenges to maintain consistent and reliable operations.
- Integrate with big data and cloud technologies: Connect NiFi with databases, cloud platforms (AWS, Azure, OCI), and big data services (e.g., Kafka, Elasticsearch, S3, SQS/SNS) to enable seamless end-to-end data movement.
- Establish best practices: Define and enforce NiFi development, deployment, and governance standards, ensuring alignment with enterprise data management policies and DevOps practices.
- Documentation and knowledge sharing: Develop comprehensive documentation, including data flow mappings, architectural diagrams, and SOPs, to support scalability and continuity across teams.
- Cross-functional collaboration: Partner closely with data architects, data engineers, and application developers to translate complex business requirements into robust technical implementations.
- Mentorship and leadership: Provide technical guidance, conduct code reviews, and mentor junior team members to enhance team capability and foster a culture of continuous improvement.
Requirements
- Proven experience designing, developing, and managing complex NiFi data flow solutions in large-scale, enterprise environments.
- Deep understanding of NiFi architecture, including processors, configurations, clustering, and NiFi Registry.
- Proficiency in Java and Python for developing custom NiFi processors and automation scripts.
- Experience managing relational and NoSQL databases (e.g., PostgreSQL, Elasticsearch, DynamoDB) and writing optimized queries.
- Strong background in real-time streaming and REST API integration for dynamic data movement.
- Hands-on experience with cloud platforms (AWS, Azure, OCI) and related data services.
- Ability to analyze complex data challenges, identify root causes, and implement effective, sustainable solutions.
- Excellent communication and collaboration skills, with the ability to clearly convey technical concepts to diverse audiences., * HS diploma plus 12 - 16+ years of relevant experience
- BS degree plus 8 - 12+ years of relevant experience
- Master's degree plus 6 - 10+ years of relevant experience
- Active TS/SCI with polygraph required., * Demonstrated experience deploying and managing ETL solutions within AWS environments, leveraging native cloud data tools and automation frameworks.