0

I'm using Apache NiFi 2.0.0, which unfortunately does not include the PutHDFS processor. My project requires this version of NiFi due to its integration capabilities with Python scripting, so upgrading to a different version with PutHDFS is not an option.

I need to transfer data from NiFi to HDFS, and I'm looking for an alternative way to do this directly within NiFi 2.0.0. Does anyone have suggestions for accomplishing this? Here are some details of what I’m considering:

Python Scripted Processors: Given that NiFi 2.0.0 has Python scripting support, is there a way to leverage this to create a custom solution that writes to HDFS?

Other Processors or Workarounds: If there are alternative processors or methods to get data into HDFS (e.g., using ExecuteStreamCommand to run HDFS CLI commands or any HTTP-based processors), I’d be interested in those as well.

Any guidance on how to handle this or examples would be greatly appreciated!

1 Answer 1

Reset to default
3

You can find more details on what components have been removed in NiFi 2 here: https://cwiki.apache.org/confluence/display/NIFI/Deprecated+Components+and+Features

Also for reference: https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance

For HDFS components specifically, there are not packaged in the convenience binary file (ie. nifi-2.0.0-bin.zip) but the components are still built and made available through Maven repositories. You can download the appropriate NARs as needed:

2
  • Thanks its working, what would be the process if basic processor is missing like ConvertExceltoCSVProcessor is also missing, its not as straight forward to add just nar file.
    – Filbadeha
    Commented Nov 7 at 10:47
  • 2
    As mentioned on the Deprecated Components page, for ConvertExceltoCSVProcessor, the recommended replacement is to use ConvertRecord with the ExcelReader and the CSVWriter.
    – Pierre Villard
    Commented Nov 7 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.