AbstractRecordProcessor

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

AbstractRecordProcessor

Chris Herrera
Hi All,

This is my second attempt for some reason my email client likes to defer to my non-registered email to write to the list, (I doubt it happened but if you receive this twice, many apologies).

I was starting to work on the HBase record processors, and while looking around found the AbstractRecordProcessor in org.apache.nifi.processors.standard. Best I can tell, this is currently implemented in UpdateRecord and ConvertRecord. However, AbstractRecordProcessor extends AbstractProcessor with the addition record-specific logic, thus making it a slightly better starting point from which to create a record-based processor. I was wondering wether it makes sense to move the AbstractRecordProcessor to  nifi-api from where it currently lives int the nifi-nar-bundles (under the standard processors).

As a follow on to this, does it make sense to create a maven archetype that basically has the shell of a record based processor (essentially a light implementation of AbstractRecordProcessor)?

Thanks!
Chris
Reply | Threaded
Open this post in threaded view
|

Re: AbstractRecordProcessor

Bryan Bende
Hi Chris,

I think a good place for an abstract record processor would be in the
standard record utils under nifi-nar-bundles/nifi-extension-utils:

https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-standard-record-utils

We've taken a similar approach for other abstract processors by putting
them in nifi-processor-utils which is also under nifi-extension-utils, but
since this depends on the record API I would lean towards the standard
record utils.

For an archetype, I am on the fence about whether maintaining a separate
archetype is worth it. We could also just document the maven dependcies
that should be added to work with records when using a project generated
from the standard archetype? With that plus the abstract record processor
it might be easy enough for people to get going.

-Bryan

On Sat, Jun 17, 2017 at 10:22 AM Chris Herrera <[hidden email]>
wrote:

> Hi All,
>
> This is my second attempt for some reason my email client likes to defer
> to my non-registered email to write to the list, (I doubt it happened but
> if you receive this twice, many apologies).
>
> I was starting to work on the HBase record processors, and while looking
> around found the AbstractRecordProcessor in
> org.apache.nifi.processors.standard. Best I can tell, this is currently
> implemented in UpdateRecord and ConvertRecord. However,
> AbstractRecordProcessor extends AbstractProcessor with the addition
> record-specific logic, thus making it a slightly better starting point from
> which to create a record-based processor. I was wondering wether it makes
> sense to move the AbstractRecordProcessor to  nifi-api from where it
> currently lives int the nifi-nar-bundles (under the standard processors).
>
> As a follow on to this, does it make sense to create a maven archetype
> that basically has the shell of a record based processor (essentially a
> light implementation of AbstractRecordProcessor)?
>
> Thanks!
> Chris
>
--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: AbstractRecordProcessor

Joe Witt
+1 to bryans comments

On Sat, Jun 17, 2017 at 8:18 AM, Bryan Bende <[hidden email]> wrote:

> Hi Chris,
>
> I think a good place for an abstract record processor would be in the
> standard record utils under nifi-nar-bundles/nifi-extension-utils:
>
> https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-standard-record-utils
>
> We've taken a similar approach for other abstract processors by putting
> them in nifi-processor-utils which is also under nifi-extension-utils, but
> since this depends on the record API I would lean towards the standard
> record utils.
>
> For an archetype, I am on the fence about whether maintaining a separate
> archetype is worth it. We could also just document the maven dependcies
> that should be added to work with records when using a project generated
> from the standard archetype? With that plus the abstract record processor
> it might be easy enough for people to get going.
>
> -Bryan
>
> On Sat, Jun 17, 2017 at 10:22 AM Chris Herrera <[hidden email]>
> wrote:
>
>> Hi All,
>>
>> This is my second attempt for some reason my email client likes to defer
>> to my non-registered email to write to the list, (I doubt it happened but
>> if you receive this twice, many apologies).
>>
>> I was starting to work on the HBase record processors, and while looking
>> around found the AbstractRecordProcessor in
>> org.apache.nifi.processors.standard. Best I can tell, this is currently
>> implemented in UpdateRecord and ConvertRecord. However,
>> AbstractRecordProcessor extends AbstractProcessor with the addition
>> record-specific logic, thus making it a slightly better starting point from
>> which to create a record-based processor. I was wondering wether it makes
>> sense to move the AbstractRecordProcessor to  nifi-api from where it
>> currently lives int the nifi-nar-bundles (under the standard processors).
>>
>> As a follow on to this, does it make sense to create a maven archetype
>> that basically has the shell of a record based processor (essentially a
>> light implementation of AbstractRecordProcessor)?
>>
>> Thanks!
>> Chris
>>
> --
> Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: AbstractRecordProcessor

Chris Herrera
In reply to this post by Chris Herrera
Excellent, thanks for the feedback guys!


Regards,Chris
-------- Original message --------From: Joe Witt <[hidden email]> Date: 6/17/17  10:20 AM  (GMT-06:00) To: [hidden email] Subject: Re: AbstractRecordProcessor
+1 to bryans comments

On Sat, Jun 17, 2017 at 8:18 AM, Bryan Bende <[hidden email]> wrote:

> Hi Chris,
>
> I think a good place for an abstract record processor would be in the
> standard record utils under nifi-nar-bundles/nifi-extension-utils:
>
> https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-standard-record-utils
>
> We've taken a similar approach for other abstract processors by putting
> them in nifi-processor-utils which is also under nifi-extension-utils, but
> since this depends on the record API I would lean towards the standard
> record utils.
>
> For an archetype, I am on the fence about whether maintaining a separate
> archetype is worth it. We could also just document the maven dependcies
> that should be added to work with records when using a project generated
> from the standard archetype? With that plus the abstract record processor
> it might be easy enough for people to get going.
>
> -Bryan
>
> On Sat, Jun 17, 2017 at 10:22 AM Chris Herrera <[hidden email]>
> wrote:
>
>> Hi All,
>>
>> This is my second attempt for some reason my email client likes to defer
>> to my non-registered email to write to the list, (I doubt it happened but
>> if you receive this twice, many apologies).
>>
>> I was starting to work on the HBase record processors, and while looking
>> around found the AbstractRecordProcessor in
>> org.apache.nifi.processors.standard. Best I can tell, this is currently
>> implemented in UpdateRecord and ConvertRecord. However,
>> AbstractRecordProcessor extends AbstractProcessor with the addition
>> record-specific logic, thus making it a slightly better starting point from
>> which to create a record-based processor. I was wondering wether it makes
>> sense to move the AbstractRecordProcessor to  nifi-api from where it
>> currently lives int the nifi-nar-bundles (under the standard processors).
>>
>> As a follow on to this, does it make sense to create a maven archetype
>> that basically has the shell of a record based processor (essentially a
>> light implementation of AbstractRecordProcessor)?
>>
>> Thanks!
>> Chris
>>
> --
> Sent from Gmail Mobile