NiFi RecordPath Question

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

NiFi RecordPath Question

David Nahoopii
https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#filters


Apologies in advance - I'm sure this email alias isn't used for questions, but I did see it at the top of the page in the link above.
Is it possible to do any AND condition with using record path?



Given the below json object, I would like to use RecordPath in an UpdateRecord processor to get the value of /loc2[*][isEmpty(./rlp) AND ./src = 'network']/acc  (meaning, src = 'network' and rlp is empty/does not exist).  Using nifi 1.2, is there no AND condition to use?  Has one been added in 1.3 or later?


Thanks and apologies in advance. - Dave


{

    "loc2": [
        {
            "acc": 92,
            "src": "gps",
            "ll": {
                "lat": 41.83173031,
                "lon": -88.09725264
            },
            "sol": 1498861615000,
            "brn": 307.7,
            "alt": 191,
            "spd": 2.48reddi
        },
        {
            "acc": 18.088,
            "src": "network",
            "ll": {
                "lat": 41.8317428,
                "lon": -88.096802
            },
            "sol": 1498865950344,
            "alt": 193.9
        },
        {
            "acc": 18.088,
            "src": "network",
            "ll": {
                "lat": 41.8317428,
                "lon": -88.096802
            },
            "sol": 1498865950344,
            "alt": 193.9,
            "rlp": "passive"
        },

        {
            "acc": 20,
            "src": "fused",
            "ll": {
                "lat": 41.8317428,
                "lon": -88.096802
            },
            "sol": 1498867975640,
            "alt": 0,
            "spd": 0
        }
    ],


}
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi RecordPath Question

Andy LoPresto-2
Hi David,

This is absolutely the correct place to ask these questions. There is also a mailing list more focused on the user experience and less of the development discussion at [hidden email] which may help as well. 

Mark Payne contributed most of the record processing code, and has written about it here [1] and here [2], and Bryan Bende has also written a tutorial [3] on using it. I am far from a record path expert, but have you tried simply combining the predicates like so? (I don’t have a 1.2 instance available right now to test with). 

 /loc2[*][isEmpty(./rlp)][./src = 'network']/acc

If this doesn’t work, hopefully Mark or another user can chime in with a suggestion, or if it is not currently possible, you can open a Jira ticket requesting this feature. 


Andy LoPresto
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Aug 9, 2017, at 3:51 PM, David Nahoopii <[hidden email]> wrote:

https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#filters


Apologies in advance - I'm sure this email alias isn't used for questions, but I did see it at the top of the page in the link above.
Is it possible to do any AND condition with using record path?



Given the below json object, I would like to use RecordPath in an UpdateRecord processor to get the value of /loc2[*][isEmpty(./rlp) AND ./src = 'network']/acc  (meaning, src = 'network' and rlp is empty/does not exist).  Using nifi 1.2, is there no AND condition to use?  Has one been added in 1.3 or later?


Thanks and apologies in advance. - Dave


{

   "loc2": [
       {
           "acc": 92,
           "src": "gps",
           "ll": {
               "lat": 41.83173031,
               "lon": -88.09725264
           },
           "sol": 1498861615000,
           "brn": 307.7,
           "alt": 191,
           "spd": 2.48reddi
       },
       {
           "acc": 18.088,
           "src": "network",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498865950344,
           "alt": 193.9
       },
       {
           "acc": 18.088,
           "src": "network",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498865950344,
           "alt": 193.9,
           "rlp": "passive"
       },

       {
           "acc": 20,
           "src": "fused",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498867975640,
           "alt": 0,
           "spd": 0
       }
   ],


}


signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NiFi RecordPath Question

Mark Payne
Hi David,

Just wanted to chime in that what Andy describes below should indeed work to provide an 'AND'
type of functionality. If you have problems getting this to work, please let us know!

Thanks
-Mark


On Aug 9, 2017, at 9:52 PM, Andy LoPresto <[hidden email]<mailto:[hidden email]>> wrote:

Hi David,

This is absolutely the correct place to ask these questions. There is also a mailing list more focused on the user experience and less of the development discussion at [hidden email]<mailto:[hidden email]> which may help as well.

Mark Payne contributed most of the record processing code, and has written about it here [1] and here [2], and Bryan Bende has also written a tutorial [3] on using it. I am far from a record path expert, but have you tried simply combining the predicates like so? (I don’t have a 1.2 instance available right now to test with).

 /loc2[*][isEmpty(./rlp)][./src = 'network']/acc

If this doesn’t work, hopefully Mark or another user can chime in with a suggestion, or if it is not currently possible, you can open a Jira ticket requesting this feature.

[1] https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
[2] https://blogs.apache.org/nifi/entry/real-time-sql-on-event
[3] http://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries

Andy LoPresto
[hidden email]<mailto:[hidden email]>
[hidden email]<mailto:[hidden email]>
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Aug 9, 2017, at 3:51 PM, David Nahoopii <[hidden email]<mailto:[hidden email]>> wrote:

https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#filters


Apologies in advance - I'm sure this email alias isn't used for questions, but I did see it at the top of the page in the link above.
Is it possible to do any AND condition with using record path?



Given the below json object, I would like to use RecordPath in an UpdateRecord processor to get the value of /loc2[*][isEmpty(./rlp) AND ./src = 'network']/acc  (meaning, src = 'network' and rlp is empty/does not exist).  Using nifi 1.2, is there no AND condition to use?  Has one been added in 1.3 or later?


Thanks and apologies in advance. - Dave


{

   "loc2": [
       {
           "acc": 92,
           "src": "gps",
           "ll": {
               "lat": 41.83173031,
               "lon": -88.09725264
           },
           "sol": 1498861615000,
           "brn": 307.7,
           "alt": 191,
           "spd": 2.48reddi
       },
       {
           "acc": 18.088,
           "src": "network",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498865950344,
           "alt": 193.9
       },
       {
           "acc": 18.088,
           "src": "network",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498865950344,
           "alt": 193.9,
           "rlp": "passive"
       },

       {
           "acc": 20,
           "src": "fused",
           "ll": {
               "lat": 41.8317428,
               "lon": -88.096802
           },
           "sol": 1498867975640,
           "alt": 0,
           "spd": 0
       }
   ],


}


Loading...