Questions about record processors

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions about record processors

Mika Borner
Hi

Been playing with the new record processors (great so far!). Now I have
two questions about usage.

1. Let's say I have an Update Record processor with a GrokReader and a
JsonRecordSetWriter manipulating the record. This works fine. Now I want
to further process the records with a LookupRecord and/or a QueryRecord
processor. In my case, setting the Record Reader for chained record
processors to JsonTreeReader seems to work. Just wondering if this is
the right thing to do, as there are only examples with a single record
processor.

2. I haven't figured out yet, how the LookupRecord processor
configuration has to be done. My lookup table contains two columns and
one row

foo, bar
foovalue1, barvalue1

The SimpleCsvFileLookupService has "Lookup Key Column" set to "foo" and
"Lookup Value Column" to "bar".

The LookupRecord processor has "Result RecordPath" set to //bar. The
property "key" is set to //foo. Here I'm not sure if the property name
"key" is correct. Any other value causes an error. Unfortunately the
lookup doesn't work and I also couldn't find any examples in the
documentation.

Thanks for your help!

Mika>






Reply | Threaded
Open this post in threaded view
|

Re: Questions about record processors

Mika Borner
Answering the second question myself.

The record paths had one slash too much /bar and /foo instead of //bar
and //foo.


On 06/11/2017 10:06 PM, Mika Borner wrote:

> Hi
>
> Been playing with the new record processors (great so far!). Now I
> have two questions about usage.
>
> 1. Let's say I have an Update Record processor with a GrokReader and a
> JsonRecordSetWriter manipulating the record. This works fine. Now I
> want to further process the records with a LookupRecord and/or a
> QueryRecord processor. In my case, setting the Record Reader for
> chained record processors to JsonTreeReader seems to work. Just
> wondering if this is the right thing to do, as there are only examples
> with a single record processor.
>
> 2. I haven't figured out yet, how the LookupRecord processor
> configuration has to be done. My lookup table contains two columns and
> one row
>
> foo, bar
> foovalue1, barvalue1
>
> The SimpleCsvFileLookupService has "Lookup Key Column" set to "foo"
> and "Lookup Value Column" to "bar".
>
> The LookupRecord processor has "Result RecordPath" set to //bar. The
> property "key" is set to //foo. Here I'm not sure if the property name
> "key" is correct. Any other value causes an error. Unfortunately the
> lookup doesn't work and I also couldn't find any examples in the
> documentation.
>
> Thanks for your help!
>
> Mika>
>
>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Questions about record processors

Joe Witt
Mika

And yes to question one.  What you're doing us fine.  There will also soon be an option for the writer to just infer its schema from the reader which may be useful in simplifying the config more.

Glad you are finding these useful.  If you'd like to contribute a sample flow that does this we have a wiki page for such things.  I'm sure others would find it helpful.

Thanks
Joe

On Jun 11, 2017 1:20 PM, "Mika Borner" <[hidden email]> wrote:
Answering the second question myself.

The record paths had one slash too much /bar and /foo instead of //bar and //foo.


On 06/11/2017 10:06 PM, Mika Borner wrote:
Hi

Been playing with the new record processors (great so far!). Now I have two questions about usage.

1. Let's say I have an Update Record processor with a GrokReader and a JsonRecordSetWriter manipulating the record. This works fine. Now I want to further process the records with a LookupRecord and/or a QueryRecord processor. In my case, setting the Record Reader for chained record processors to JsonTreeReader seems to work. Just wondering if this is the right thing to do, as there are only examples with a single record processor.

2. I haven't figured out yet, how the LookupRecord processor configuration has to be done. My lookup table contains two columns and one row

foo, bar
foovalue1, barvalue1

The SimpleCsvFileLookupService has "Lookup Key Column" set to "foo" and "Lookup Value Column" to "bar".

The LookupRecord processor has "Result RecordPath" set to //bar. The property "key" is set to //foo. Here I'm not sure if the property name "key" is correct. Any other value causes an error. Unfortunately the lookup doesn't work and I also couldn't find any examples in the documentation.

Thanks for your help!

Mika>