OpenWhisk Feeds provide the mechanism to bind external events sources to serverless function executions.
Want to use OpenWhisk to listen for database updates to generate usage statistics? Or write Slack bots that respond to trigger words? Or notify users when Github project changes?
Rather than manually listening for these events with an external application and calling OpenWhisk Actions through the API, OpenWhisk Feeds automate connecting external events sources with Actions.
Feeds allow users to register Triggers to be invoked when external events occur. Defining Rules that bind these Triggers to Actions, we can have Actions run for external events.
OpenWhisk Packages
Feeds are contained within and accessible through Packages.
OpenWhisk provides numerous built-in packages under the whisk.system namespace. These packages contain both public Actions and Feeds.
$ wsk package list /whisk.system
packages
/whisk.system/alarms shared
/whisk.system/cloudant shared
/whisk.system/watson shared
/whisk.system/system shared
/whisk.system/weather shared
/whisk.system/util shared
/whisk.system/slack shared
/whisk.system/samples shared
/whisk.system/github
Retrieving the package summary, the Actions and Feeds contained within are returned. Feeds are referenced by the publisher’s namespace, package and feed name, e.g. /whisk.system/alarms/alarm
$ wsk package get /whisk.system/alarms --summary
package /whisk.system/alarms: Alarms and periodic utility
(params: cron trigger_payload)
feed /whisk.system/alarms/alarm: Fire trigger when alarm occurs
The Alarm package (/whisk.system/alarms) contains a single Feed (/whisk.system/alarms/alarm) that calls the registered Trigger on a fixed schedule. Users provide the timer schedule through the cron parameter.
When creating new Triggers, users can specify a Feed source to bind their new Trigger to the external event source. Parameters from the command-line will be passed to the Feed source. The Feed provider will execute the Trigger each time an external event occurs.
$ wsk trigger create everySecond --feed /whisk.system/alarms/alarm -p cron '*/1 * * * * *' -p trigger_payload '{"vote":"Bernie"}'
This new Trigger will be invoked every second with the payload contents by the alarm Feed. Using rules, the Feed Trigger can be bound to call an Action on each invocation.
$ wsk rule create --enable alarmRule everySecond actionName
As well as using the built-in Feeds, users can create and register custom Feeds. This provides a way to integrate almost any external event source into the platform. Publishing custom Feeds within a public package will also make this event source available to all users on the systen, provided they know the package identifier.
Creating Custom Feeds
Users register new Feeds by providing a custom Action to the platform. This Action is invoked each time the Feed is bound to a new Trigger. Authentication credentials, supporting Trigger invocation through the OpenWhisk API, are passed in as invocation parameters.
This sample Action contains an outline for processing requests.
// params.lifeCycleEvent - Enum value (CREATE|DELETE)
// params.triggerName - User's Trigger identifier to call
// params.authKey - Authentication details for calling Trigger
function main (params) {
if (params.lifecycleEvent === 'CREATE') {
create(params);
} else if (params.lifecycleEvent === 'DELETE') {
remove(params)
}
return whisk.async();
}
The params argument contains the Trigger information provided by the platform and any parameters from the user during creation.
The lifeCycleEvent parameter is a string value, informing the Feed provider whether to register (CREATE) or remove (DELETE) the user’s Trigger with the event source. The Trigger identifier is passed as the triggerName parameter, with the authentication key (authKey) used for the API requests.
Feed Actions must be registered with a custom annotation (feed), allowing the platform to distinguish them from “normal” Actions. This annotation can be set during the create command.
$ wsk action create -a feed true feed_name feed_action.js
Once a custom Feed Action has been registered, users can create new Triggers using that Feed source, following the steps above.
MQTT Feeds
The “Internet of Things” is often cited as a common usecase for serverless platforms. Solutions are often event-driven and stateless, e.g. wait for data from this device, do some processing and then store the results in this database.
MQTT is a lightweight publish-subscribe messaging protocol, commonly used for edge of network device-to-device communication.
Bridging MQTT messages to OpenWhisk Actions can be achieved by creating a new Feed provider. This provider would subscribe to message topics and execute registered Triggers with incoming messages.
The custom feed provider would need to establish and maintain long-lived MQTT connections, waiting for messages to arrive. This requirements means the Feed provider needs an external service to handle managing these connections, it won’t be possible within the Feed Action.
This feed provider service is implemented using Node.js, using Cloudant for the database. The service listens for HTTP requests, with Trigger registration details, from the Feed Action. The Node.js MQTT library is used to subscribe to registered topics. When messages are received, the OpenWhisk client library is used to invoke the Trigger remotely, passing the message contents as event parameters.
This service provider is packaged using Docker.
Pushing this image into the IBM Containers registry, the Feed provider can be started on IBM Bluemix using the Containers service.
$ docker build -t USERNAME/mqtt_feed_provider .
$ docker tag USERNAME/mqtt_feed_provider registry.ng.bluemix.net/USERNAME/mqtt_feed_provider
$ docker push registry.ng.bluemix.net/USERNAME/mqtt_feed_provider
Registering Feeds
With the Feed service provider running, the Feed Action can be deployed.
The Feed will be registered under the name, mqtt_feed_provider, in a custom package, mqtt.
Using the –shared command-line flag, the Feed package can be registered as a public package. Feeds and Actions within public packages are visible to every system user.
Rather than hardcoding the service provider location within the Feed Action, this configuration value will be accessible as a package parameter. This can be updated at runtime with modifying the Feed Action source.
$ wsk package create --shared -p provider_endpoint "http://CONTAINER_IP:3000/mqtt" mqtt
$ wsk package update mqtt -a description 'MQTT topic feed. Messages received on broker topic as passed to triggers"
Having created the package, we can add the Feed Action, using the custom attribute to denote this is a Feed Action.
$ wsk action create -a feed true mqtt/mqtt_feed mqtt_feed.js
Once the Feed has been registered, it can be referenced when creating new Triggers.
$ wsk trigger create feed_trigger --feed /james.thomas@uk.ibm.com_dev/mqtt/mqtt_feed -p topic 'whiskers' -p url 'mqtt://test.mosca.io'
MQTT broker url and topic name are passed as Trigger parameters, using the -p flags. These values are included within the invocation arguments to the Feed Action, shown below.
var params = {
authKey: 'USERNAME:PASSWORD',
url: 'mqtt://test.mosca.io',
provider_endpoint: 'http://CONTAINER_IP:3000/mqtt',
topic: 'whiskers',
lifecycleEvent: 'CREATE',
triggerName: '/james.thomas@uk.ibm.com_dev/feed_trigger'
}
Once the Feed service provider has connected to the broker and subscribed to the topic, incoming messages will register as Trigger events invocations for the public_feed Trigger.
Using this custom Feed, users can easily connect MQTT messages to OpenWhisk Actions.
Github Project
Source code for this custom OpenWhisk Feed is available here. The project contains the Feed Action and Provider service. The README contains the deployment and usage instructions.