Proposal: Messaging nodes

[binnes]

Messaging systems often provide the backbone to event driven architectures, so we should support the IBM messaging services in Node-RED, as Node-RED works particularly well in event driven scenarios.

MQ and Event Streams are the 2 primary messaging services in the Integration category of the IBM Cloud Catalog.

There is little support for either IBM service in Node-RED currently. However, as Event Streams is built on the Open Source Kafka application, any of the 13 available Kafka nodes should be able to be used with the Event Streams service (though authentication, etc... has to be configured manually).

The MQ service is less well supported with no active nodes available, though the IBM MQ service can support the MQTT and AMQP protocols and there are nodes for those protocols available.

Items to resolve:

• Confirm we want to support both IBM messaging services (MQ and Event Streams)
• Identify base level of functionality the support should have
• Identify if any of the available nodes offer the required level of functionality
• Establish outline of work needed based on the above points

Note: The IBM IoT Platform also supports MQTT, but is not included in this proposal

[knolleary]

I know that @dalelane has played a bit with the kafka side of things in Node-RED. Would be good to get his input here.

[johnwalicki]

We should not forget to mention the MQTT based messaging Node-RED nodes created in the past by the Watson IoT engineering teams.

• node-red-contrib-ibm-watson-iot
• node-red-contrib-scx-ibmiotapp
• node-red-contrib-iot-virtual-device
• node-red-contrib-ibm-wiotp-device-ops

Many of those need a refresh.

[binnes]

We should not forget to mention the MQTT based messaging Node-RED nodes created in the past by the Watson IoT engineering teams.

• node-red-contrib-ibm-watson-iot
• node-red-contrib-scx-ibmiotapp
• node-red-contrib-iot-virtual-device
• node-red-contrib-ibm-wiotp-device-ops

Many of those need a refresh.

I was keeping the IBM IoT nodes separate from the IBM messaging services (those in the integration category in the cloud catalog). If we want to support the IBM IoT Platform over the open source MQTT protocol then we should raise another proposal for those nodes?

[johnwalicki]

Agree that the IBM IoT nodes can be a separate GitHub Issue, just wanted to mention it here in the context of Messaging services.

[dalelane]

There is little support for either IBM service in Node-RED currently. However, as Event Streams is built on the Open Source Kafka application, any of the 13 available Kafka nodes should be able to be used with the Event Streams service (though authentication, etc... has to be configured manually).

The challenge with this is that Event Streams has always required authentication for Kafka clients, and some of the Node-RED Kafka nodes haven't supported authentication.

I added auth support to one of the more popular nodes for just this reason in hjespers/node-red-contrib-rdkafka#14

With the OpenShift flavour of Event Streams, the new version 10.0.0 released this month allows administrators to disable authentication - that should make it compatible with almost any Kafka nodes (I'm speculating to be honest, I've not tested all of them!) - so things are a little easier there now. But for the IBM Cloud Event Streams service, that's obviously always going to require auth.

[dalelane]

And to add a more positive note, for Node-RED nodes that have a way to provide auth credentials, working with Event Streams becomes super easy! https://dalelane.co.uk/blog/?p=3732

[chughts]

MQ over REST nodes do exist - https://flows.nodered.org/node/node-red-contrib-mqrest-client
GET / PUT is supported.
PUBLISH and Admin REST capabilities need to be added.