Clarification on "periodic task" processor

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Clarification on "periodic task" processor

Russell Bateman-2
I'm doing something I've never done before in a custom processor. I'm
writing a processor that will awaken every /n/ minutes, query a
database, perform work as indicated by the result of the query, then go
back to sleep for /n/ minutes.

This processor will never expect a flowfile because it doesn't work on
flowfile attributes or content. It may, I think, emit some flowfiles
downstream.

Will onTrigger()be called without a flowfile? That point seems unclear
in the Developer Guide. Or is there some other lifecycle, like
@OnScheduledthat I should expect to use for this?

Thanks,

Russ
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarification on "periodic task" processor

Andy LoPresto-2
Russell,

My understanding is that onTrigger() will be called based on the scheduling of the processor, regardless of if there is an incoming flowfile. A couple of examples support this line of thought — “source” processors like GetFile, GenerateFlowFile, etc. run according to their schedule even though they will never receive a flowfile. Similarly, in ExecuteScript, the first task is always to attempt to retrieve a flowfile from the session and then yield execution if none is returned. 

We may be able to add more explicit information to the Developer Guide, but I would suggest looking at Performing the Work [1] and When Processors Are Triggered [2] as they cover this topic. It seems in your case, your processor will either have no incoming connections, or you’ll need to add the @TriggerWhenEmpty annotation to your processor. 




Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Jun 8, 2017, at 3:45 PM, Russell Bateman <[hidden email]> wrote:

I'm doing something I've never done before in a custom processor. I'm writing a processor that will awaken every /n/ minutes, query a database, perform work as indicated by the result of the query, then go back to sleep for /n/ minutes.

This processor will never expect a flowfile because it doesn't work on flowfile attributes or content. It may, I think, emit some flowfiles downstream.

Will onTrigger()be called without a flowfile? That point seems unclear in the Developer Guide. Or is there some other lifecycle, like @OnScheduledthat I should expect to use for this?

Thanks,

Russ


signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarification on "periodic task" processor

Russell Bateman-2
Thanks, Andy. I guess point 2, "the processor has no in-coming
connections", would answer this, but I'll annotate too--it's good
"documentation" to have the annotation.


On 06/08/2017 05:03 PM, Andy LoPresto wrote:

> Russell,
>
> My understanding is that onTrigger() will be called based on the
> scheduling of the processor, regardless of if there is an incoming
> flowfile. A couple of examples support this line of thought — “source”
> processors like GetFile, GenerateFlowFile, etc. run according to their
> schedule even though they will never receive a flowfile. Similarly, in
> ExecuteScript, the first task is always to attempt to retrieve a
> flowfile from the session and then yield execution if none is returned.
>
> We may be able to add more explicit information to the Developer
> Guide, but I would suggest looking at Performing the Work [1] and When
> Processors Are Triggered [2] as they cover this topic. It seems in
> your case, your processor will either have no incoming connections, or
> you’ll need to add the @TriggerWhenEmpty annotation to your processor.
>
> [1]
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#performing-the-work
> [2]
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#when-processors-are-triggered
>
>
>
> Andy LoPresto
> [hidden email] <mailto:[hidden email]>
> /[hidden email] <mailto:[hidden email]>/
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>> On Jun 8, 2017, at 3:45 PM, Russell Bateman <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> I'm doing something I've never done before in a custom processor. I'm
>> writing a processor that will awaken every /n/ minutes, query a
>> database, perform work as indicated by the result of the query, then
>> go back to sleep for /n/ minutes.
>>
>> This processor will never expect a flowfile because it doesn't work
>> on flowfile attributes or content. It may, I think, emit some
>> flowfiles downstream.
>>
>> Will onTrigger()be called without a flowfile? That point seems
>> unclear in the Developer Guide. Or is there some other lifecycle,
>> like @OnScheduledthat I should expect to use for this?
>>
>> Thanks,
>>
>> Russ
>

Loading...