New processor location

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

New processor location

Michael Hogue
Hello all,

   In thinking about adding a new processor, what considerations should be
made in order to determine whether the processor belongs in
nifi-standard-processors or in its own bundle?

   From the hipchat the following general guidelines were laid out:

   - If it's a proprietary or obscure service, it belongs in its own bundle.

   - If it's a very common/standard format and it doesn't bring with it an
   enormous set of dependencies, then it can go in standard.

    I think that's a pretty solid set of general rules, but are there
additional nuances that should be considered? For example, I've submitted
PR1927 [1] to add InvokeGRPC as a standard processor, and while gRPC is
becoming a more common transport mechanism, it adds the first occurrence of
code generation in nifi-standard-processors.

   It was suggested in the hipchat that we formalize and agree on the set
of things to consider when deciding on where to place a new processor
implementation. Once agreed upon, we should add it to the contributor guide
[2]. Thanks, Tony and Andy, for the feedback.

Regards,
Mike

[1] https://github.com/apache/nifi/pull/1927
[2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: New processor location

Bryan Bende
Hi Michael,

Those two points are definitely good ones to help make the decision.
For the second point, it doesn't necessarily have to be an enormous
set of dependencies, it could be just one dependency that the new
processors need that is in conflict with what the standard processors
are already using.

For this case it looks like about 20 JARs get brought in by the gRPC
and Netty dependencies,...I don't think any of them necessarily cause
a problem for standard processors, but with that plus the code
generation, I would learn towards a separate NAR.

A separate NAR also helps from a deployment perspective... its easier
to give someone a separate NAR to add to their existing install, and
when the extension registry comes about it would be easier to publish
a gRPC NAR rather than have to publish the whole standard NAR when
needing to update the gRPC processors. There are even processors that
are already in the standard NAR that should probably be broken out
once we have an extension registry.

Just my two-cents, hopefully others chime in as well.

Thanks,

Bryan


On Mon, Jun 19, 2017 at 6:01 PM, Michael Hogue
<[hidden email]> wrote:

> Hello all,
>
>    In thinking about adding a new processor, what considerations should be
> made in order to determine whether the processor belongs in
> nifi-standard-processors or in its own bundle?
>
>    From the hipchat the following general guidelines were laid out:
>
>    - If it's a proprietary or obscure service, it belongs in its own bundle.
>
>    - If it's a very common/standard format and it doesn't bring with it an
>    enormous set of dependencies, then it can go in standard.
>
>     I think that's a pretty solid set of general rules, but are there
> additional nuances that should be considered? For example, I've submitted
> PR1927 [1] to add InvokeGRPC as a standard processor, and while gRPC is
> becoming a more common transport mechanism, it adds the first occurrence of
> code generation in nifi-standard-processors.
>
>    It was suggested in the hipchat that we formalize and agree on the set
> of things to consider when deciding on where to place a new processor
> implementation. Once agreed upon, we should add it to the contributor guide
> [2]. Thanks, Tony and Andy, for the feedback.
>
> Regards,
> Mike
>
> [1] https://github.com/apache/nifi/pull/1927
> [2] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide
Loading...