NiFi 1.2.0 REST API problem

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

NiFi 1.2.0 REST API problem

Raymond Rogers-2
We have a node.js service that automatically creates & manages NiFi groups using the REST API which works great in NiFi 1.1.1.  We are upgrading our NiFi instances to 1.2.0 and I have found that some of the processors are exhibiting odd behavior.

We have a flow the connects to the Outlook 365 OWA service generates an access token and then uses that token in two different InvokeHTTP processors.  The first processor always works and the second always returns an HTTP error 401.

If I delete and manually re-add the InvokeHTTP processor with the same configuration it always works.

If I export this flow from the NiFi web interface and then re-import it, only fixing the SSL context service, it works every time.

Using our node.js service to create the exact same flow in NiFi 1.1.1 it always works.

Thanks,
Raymond
Reply | Threaded
Open this post in threaded view
|

Re: NiFi 1.2.0 REST API problem

Matt Gilman
Raymond,

When it's in a state that is not working are there any bulletins on the second processor? When it's in that state and you view the configuration details for that processor, do the properties look correct and the same as when you manually re-add the processor through the UI? Specifically, I'm wondering about the SSL Context Service since you mentioned fixing that after an export/import process resolves the issue.

Any other issues in the logs/nifi-app.log or the logs/nifi-user.log?

Thanks

Matt

On Thu, Jun 8, 2017 at 11:59 AM, Raymond Rogers <[hidden email]> wrote:
We have a node.js service that automatically creates & manages NiFi groups using the REST API which works great in NiFi 1.1.1.  We are upgrading our NiFi instances to 1.2.0 and I have found that some of the processors are exhibiting odd behavior.

We have a flow the connects to the Outlook 365 OWA service generates an access token and then uses that token in two different InvokeHTTP processors.  The first processor always works and the second always returns an HTTP error 401.

If I delete and manually re-add the InvokeHTTP processor with the same configuration it always works.

If I export this flow from the NiFi web interface and then re-import it, only fixing the SSL context service, it works every time.

Using our node.js service to create the exact same flow in NiFi 1.1.1 it always works.

Thanks,
Raymond

Reply | Threaded
Open this post in threaded view
|

Re: NiFi 1.2.0 REST API problem

Raymond Rogers-2
No bulletins on any of the processors.  All of the output flow-files have 0 bytes and the error 401 in the attributes.
All of the properties look correct and I can copy the values from the non-working to the manually created processor and it works fine.
When you export the SSL context service and re-import it you have to reset the password on the trust store and that is the only change I am making.

I will need to dig into the nifi logs to check for any errors there.

On Thu, Jun 8, 2017 at 11:24 AM, Matt Gilman <[hidden email]> wrote:
Raymond,

When it's in a state that is not working are there any bulletins on the second processor? When it's in that state and you view the configuration details for that processor, do the properties look correct and the same as when you manually re-add the processor through the UI? Specifically, I'm wondering about the SSL Context Service since you mentioned fixing that after an export/import process resolves the issue.

Any other issues in the logs/nifi-app.log or the logs/nifi-user.log?

Thanks

Matt

On Thu, Jun 8, 2017 at 11:59 AM, Raymond Rogers <[hidden email]> wrote:
We have a node.js service that automatically creates & manages NiFi groups using the REST API which works great in NiFi 1.1.1.  We are upgrading our NiFi instances to 1.2.0 and I have found that some of the processors are exhibiting odd behavior.

We have a flow the connects to the Outlook 365 OWA service generates an access token and then uses that token in two different InvokeHTTP processors.  The first processor always works and the second always returns an HTTP error 401.

If I delete and manually re-add the InvokeHTTP processor with the same configuration it always works.

If I export this flow from the NiFi web interface and then re-import it, only fixing the SSL context service, it works every time.

Using our node.js service to create the exact same flow in NiFi 1.1.1 it always works.

Thanks,
Raymond


Reply | Threaded
Open this post in threaded view
|

Re: NiFi 1.2.0 REST API problem

Matt Gilman
Raymond,

If you enable debug level logging, I believe that InvokeHTTP will log the request and response. It may be helpful in diagnosing this issue. I think you could just set the bulletin level to DEBUG to see these as messages as bulletins. Additionally, you can update your conf/logback.xml to enable DEBUG messages for org.apache.nifi.processors.standard.InvokeHTTP to see these messages in your logs/nifi-app.log.

Thanks

Matt

On Thu, Jun 8, 2017 at 1:01 PM, Raymond Rogers <[hidden email]> wrote:
No bulletins on any of the processors.  All of the output flow-files have 0 bytes and the error 401 in the attributes.
All of the properties look correct and I can copy the values from the non-working to the manually created processor and it works fine.
When you export the SSL context service and re-import it you have to reset the password on the trust store and that is the only change I am making.

I will need to dig into the nifi logs to check for any errors there.

On Thu, Jun 8, 2017 at 11:24 AM, Matt Gilman <[hidden email]> wrote:
Raymond,

When it's in a state that is not working are there any bulletins on the second processor? When it's in that state and you view the configuration details for that processor, do the properties look correct and the same as when you manually re-add the processor through the UI? Specifically, I'm wondering about the SSL Context Service since you mentioned fixing that after an export/import process resolves the issue.

Any other issues in the logs/nifi-app.log or the logs/nifi-user.log?

Thanks

Matt

On Thu, Jun 8, 2017 at 11:59 AM, Raymond Rogers <[hidden email]> wrote:
We have a node.js service that automatically creates & manages NiFi groups using the REST API which works great in NiFi 1.1.1.  We are upgrading our NiFi instances to 1.2.0 and I have found that some of the processors are exhibiting odd behavior.

We have a flow the connects to the Outlook 365 OWA service generates an access token and then uses that token in two different InvokeHTTP processors.  The first processor always works and the second always returns an HTTP error 401.

If I delete and manually re-add the InvokeHTTP processor with the same configuration it always works.

If I export this flow from the NiFi web interface and then re-import it, only fixing the SSL context service, it works every time.

Using our node.js service to create the exact same flow in NiFi 1.1.1 it always works.

Thanks,
Raymond



Reply | Threaded
Open this post in threaded view
|

Re: NiFi 1.2.0 REST API problem

Raymond Rogers-2
I have not been able to find anything in the logs that is useful, but I'm not sure I enabled all of the logging that I need to.

But I did find some more information, if I restart the NiFi service the InvokeHTTP call starts working and we have two other flow patterns that have the same issue.  In each case it is always the second InvokeHTTP processor that fails.

This is how I have the logging set currently:
    <logger name="org.apache.nifi" level="INFO"/>
    <logger name="org.apache.nifi.processors" level="DEBUG"/>
    <logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO"/>
    <logger name="org.apache.nifi.controller.repository.StandardProcessSession" level="WARN" />

    <logger name="org.apache.nifi.web.api.config" level="DEBUG" additivity="false">
        <appender-ref ref="USER_FILE"/>
    </logger>

All of the other logging is set to its default value.

On Thu, Jun 8, 2017 at 12:20 PM, Matt Gilman <[hidden email]> wrote:
Raymond,

If you enable debug level logging, I believe that InvokeHTTP will log the request and response. It may be helpful in diagnosing this issue. I think you could just set the bulletin level to DEBUG to see these as messages as bulletins. Additionally, you can update your conf/logback.xml to enable DEBUG messages for org.apache.nifi.processors.standard.InvokeHTTP to see these messages in your logs/nifi-app.log.

Thanks

Matt

On Thu, Jun 8, 2017 at 1:01 PM, Raymond Rogers <[hidden email]> wrote:
No bulletins on any of the processors.  All of the output flow-files have 0 bytes and the error 401 in the attributes.
All of the properties look correct and I can copy the values from the non-working to the manually created processor and it works fine.
When you export the SSL context service and re-import it you have to reset the password on the trust store and that is the only change I am making.

I will need to dig into the nifi logs to check for any errors there.

On Thu, Jun 8, 2017 at 11:24 AM, Matt Gilman <[hidden email]> wrote:
Raymond,

When it's in a state that is not working are there any bulletins on the second processor? When it's in that state and you view the configuration details for that processor, do the properties look correct and the same as when you manually re-add the processor through the UI? Specifically, I'm wondering about the SSL Context Service since you mentioned fixing that after an export/import process resolves the issue.

Any other issues in the logs/nifi-app.log or the logs/nifi-user.log?

Thanks

Matt

On Thu, Jun 8, 2017 at 11:59 AM, Raymond Rogers <[hidden email]> wrote:
We have a node.js service that automatically creates & manages NiFi groups using the REST API which works great in NiFi 1.1.1.  We are upgrading our NiFi instances to 1.2.0 and I have found that some of the processors are exhibiting odd behavior.

We have a flow the connects to the Outlook 365 OWA service generates an access token and then uses that token in two different InvokeHTTP processors.  The first processor always works and the second always returns an HTTP error 401.

If I delete and manually re-add the InvokeHTTP processor with the same configuration it always works.

If I export this flow from the NiFi web interface and then re-import it, only fixing the SSL context service, it works every time.

Using our node.js service to create the exact same flow in NiFi 1.1.1 it always works.

Thanks,
Raymond