Apache NiFi / Data Integration Engineer

PROPERTY CONSULTANT FIN SVC
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote

Tech stack

Java
Big Data
Configuration Management
Communications Protocols
Computer Programming
Data Integration
Data Integrity
DevOps
Ethernet
Fault Tolerance
Groovy
Hadoop
Monitoring of Systems
Supervisory Control and Data Acquisition (SCADA)
JSON
Python
Modbus
Message Queuing Telemetry Transport (MQTT)
Networking Basics
Routing
Network Segmentation
OPC Unified Architecture
Shell Script
Data Streaming
Systems Integration
XML
Parquet
Data Processing
Data Ingestion
System Availability
Spark
GIT
Low Latency
Avro
Kafka
Apache Nifi
Video Streaming
Stream Processing

Job description

The Senior Apache NiFi Engineer is responsible for the design, development, administration, and operational support of Apache NiFi based data integration platforms in enterprise production environments. The role focuses on delivering secure, scalable, and high performance data ingestion and real time processing solutions, including support for industrial and IoT data integration use cases., * Administer, operate, and support Apache NiFi clusters in production environments

  • Design, develop, and maintain NiFi data flows, processors, queues, templates, and controller services
  • Perform NiFi upgrades, patching, and configuration management * Monitor system performance including throughput, latency, and failure rates
  • Troubleshoot flow failures, back pressure conditions, and node level issues
  • Implement and manage NiFi security including TLS, user access, policies, and certificates
  • Conduct root cause analysis for outages and performance degradation
  • Ensure high availability, node health, fault tolerance, and data integrity
  • Collaborate with architects, data engineers, and stakeholders to deliver reliable data integration solutions

Requirements

  • Strong hands on experience designing, developing, and deploying Apache NiFi data flows
  • In depth knowledge of NiFi processors, controller services, templates, and reporting tasks
  • Experience with data ingestion, transformation, routing, and system integration
  • Familiarity with data formats such as JSON, XML, CSV, Avro, and Parquet
  • Solid understanding of batch and real time data processing architectures IoT & Industrial Data Processing
  • Experience ingesting and processing data from industrial sensors, PLCs, SCADA systems, and ICS devices
  • Familiarity with industrial communication protocols such as MQTT, OPC UA, Modbus, DNP3, and EtherNet/IP
  • Experience handling high volume, low latency telemetry data streams
  • Understanding of industrial and ICS security requirements, including network segmentation and secure communication Programming, Platform & DevOps
  • Proficiency in Groovy, Python, or Shell scripting; Java knowledge preferred
  • Experience working in Linux/Unix environments, with knowledge of networking fundamentals and SSL/TLS
  • Exposure to Kafka, Hadoop, Spark, or similar big data and streaming technologies (preferred)
  • Experience with NiFi monitoring, logs, provenance data, and metrics
  • Familiarity with Git and CI/CD pipelines for deploying and managing NiFi flows

Apply for this position