How to Reference/Deploy RecordReader?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to Reference/Deploy RecordReader?

Cam Q. Mach

Hello Nifi Dev,

 

I have recently built a new PutKudu processor for NiFi. In this processor, I leverage RecordReader (just like PutParquet does), to read the content of the FlowFile. I can run the build successfully on my local, but couldn’t start the NiFi. And realized that the RecordReader is not being referenced/deployed properly. And so I got this error message “ClassNotFoundException: org.apache.nifi.serialization.RecordReaderFactory” in the log file. I can manually copy the RecordReader package and deploy it to the lib directory, and start the NiFi without any issue, but shouldn’t be  done this way.

I compared every file of PutParquet with my PutKudu, but couldn’t find any missing parts of it. Would you please help?

 

Thanks,

Cam Mach

Inspur USA


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to Reference/Deploy RecordReader?

Bryan Bende
Hi Cam,

Does the pom.xml of your NAR have a NAR dependency on standard-services-api-nar?

<dependency>
    <groupId>org.apache.nifi</groupId>
    <artifactId>nifi-standard-services-api-nar</artifactId>
    <type>nar</type>
</dependency>

Using parquet as an example, your processors pom should have provided
dependencies on the record modules you need to compile your code:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-processors/pom.xml#L47-L61

But at runtime they should come through a NAR dependency:

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-parquet-bundle/nifi-parquet-nar/pom.xml#L39-L43

In this case, hadoop-libraries-nar depends on
standard-services-api-nar so it is still accessible.

-Bryan


On Wed, Aug 2, 2017 at 1:02 PM, Cam Q. Mach <[hidden email]> wrote:

> Hello Nifi Dev,
>
>
>
> I have recently built a new PutKudu processor for NiFi. In this processor, I
> leverage RecordReader (just like PutParquet does), to read the content of
> the FlowFile. I can run the build successfully on my local, but couldn’t
> start the NiFi. And realized that the RecordReader is not being
> referenced/deployed properly. And so I got this error message
> “ClassNotFoundException: org.apache.nifi.serialization.RecordReaderFactory”
> in the log file. I can manually copy the RecordReader package and deploy it
> to the lib directory, and start the NiFi without any issue, but shouldn’t be
> done this way.
>
> I compared every file of PutParquet with my PutKudu, but couldn’t find any
> missing parts of it. Would you please help?
>
>
>
> Thanks,
>
> Cam Mach
>
> Inspur USA
Loading...