James Thomas

Notes on JavaScript

OpenWhisk and Node-RED

Node-RED nodes for OpenWhisk were initially released earlier this year. The nodes allowed users to manually invoke existing Actions and Triggers. This month, a new version of the package has been released providing a huge improvement in the functionality…

features

  • Users can now define new Actions using the flow editor UI, providing the source code through the inline node configuration panel.
  • Users can also modify existing Actions, with the live Action source being previewed in the node editor panel.
  • Triggers can be created and updated in the same way.
  • Both nodes allow users to view, define and modify default parameters for both Actions and Triggers.

Deploying the flow will make the modifications to Actions and Triggers live for the configured OpenWhisk platform.

example

This video shows the updated nodes being used to define a new OpenWhisk Action, invoking it in response to a message from an inject node and then making modifications to the source code.

interested?

Grab the updated NPM package to test the new features out today…

Microservices Without Servers

…is the title of my presentation about building serverless applications using OpenWhisk.

Abstract

Servers are killing your productivity. Rather than building better software for your users, you end up constantly distracted by maintaining computers. Wasn’t the “cloud” supposed to fix this? It sounded so promising until we realised it was just renting VMs in someone else’s datacenter. We couldn’t escape “servers”. Until now…

In this session, developers will learn how to build microservices without servers, using modern “serverless” cloud platforms. We’ll look at common challenges (and solutions) to building applications using “serverless” stacks. Exploring emerging “serverless” design patterns will give developers the knowledge to build application architectures using these new platforms.

This session is aimed at software developers experienced in building traditional backend web applications, who want to learn how to build microservices without servers.

Slides, Code, Videos

Slides for the talk are available here.

There’s a Github repository with resources from the talk, including a transcript, code demos and videos.

If you have questions or issues, raise an issue in the repository or send me a tweet.

Conference Sessions

This talk was first delivered at JDayLviv earlier this month. The session was recorded so the video should surface online soon. I’ll be repeating the talk at VoxxedDays Belgrade and JavaDay Kiev in September and October.

Node-RED Docker Images

This week, I’ve been helping create and publish official Docker images for the Node-RED project. Users can start Node-RED instances from these images using the following command.

1
docker run -it -p 1880:1880 nodered/node-red-docker

Node-RED is now publishing the following images to Docker Hub for each new release.

When a new version is released on NPM, an automated CI service will build, test and publish new images with the updated version tags.

The source repository for the Docker images is available at https://github.com/node-red/node-red-docker.

Background

There was a long-standing issue open with the project to provide official Docker images for the tool. Many users had already been experimenting with Node-RED and Docker.

Reviewing the community’s efforts, we wanted to create official images that made it simple for users to start Node-RED as Docker containers with minimal configuration whilst allowing for easy customisation, i.e. adding new nodes.

Docker images are created using a configuration file (Dockerfile) that lists the commands to build that image and can start by using another image as the ‘base’.

Node-RED is a Node.js application, published as an NPM module. The Node.js project publishes official Docker images which we used as our base image. These images provide an environment with the correct versions of Node.js and NPM installed.

Rather than manually copying the Node-RED source code into the container image, we used NPM to install the source code by defining a custom package.json which includes Node-RED as dependency.

package.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
    "name": "node-red-docker",
    "version": "0.14.5",
    "description": "Docker images for Node-RED",
    "main": "node_modules/node-red/red/red.js",
    "scripts": {
        "start": "node-red"
    },
    ...
    "dependencies": {
        "node-red": "0.14.5"
    },
    "engines": {
        "node": "4.*.*"
    }
}

Adding this file into the container image and then running NPM install, using the ADD and RUN commands, will retrieve the correct Node-RED version and build that into the container image.

Docker images define a default start command to run when the container is created. Using npm start for this image will parse the start script listed in the package.json file, which has been set to node-red.

Adding New Nodes

Node-RED has a huge community which produces custom nodes for everything from accessing data from a Raspberry Pi’s sensors to a Tesla car.

Additional nodes can be installed by putting the files into your user directory, which defaults to $HOME/.node-red.

Allowing users to install additional nodes without building new images is possible using Docker’s volume support. Docker data volumes can be used to share files between the container and the host system, by mounting a directory on the host as a data volume within the container.

Exposing the Node-RED user directory within the container as a data volume means users can mount this on the host system. Nodes installed into this directory, using NPM on the host system, will automatically be registered when Node-RED starts.

Within the Dockerfile for the Node-RED image, the /data directory is configured as the user directory and exported as a data volume.

Users can mount their local user directory into the container with the following command.

1
docker run -it -p 1880:1880 -v ~/.node-red:/data nodered/node-red-docker

Environment Parameters

Docker supports injecting environment parameter values into running containers, using command-line options on the host system. This is often used to configure runtime options without users having to build new container images. Node-RED’s Docker images support the following environment parameters.

Flows Configuration

User flow configurations are stored in a JSON file under the user directory. This defaults to flows.json but can be configured using an environment parameter (FLOWS) passed to the container, as shown below.

1
docker run -it -p 1880:1880 -e FLOWS=my_flows.json nodered/node-red-docker

Node Options

Node.js runtime arguments can be passed to the container using an environment parameter (NODE_OPTIONS). For example, to fix the heap size used by the Node.js garbage collector you would use the following command.

1
docker run -it -p 1880:1880 -e NODE_OPTIONS="--max_old_space_size=128" nodered/node-red-docker

Alpine Linux Image

The official Node.js Docker image uses the Debian Jessie base image. This image provides a full Linux install, which means dependent Docker images can be hundreds of megabytes in size. Node-RED’s Docker image, using this base image, is nearly 300 MB.

Reducing Docker image sizes can dramatically reduce build and deployment times.

Alpine Linux is a lightweight Linux distribution, focused on security and performance. A minimal Docker image based on Alpine Linux is only 5 MB in size!

Using the alpine-node base image, which provides an Alpine Linux environment with Node.js & NPM, in our Dockerfiles reduces the resulting image file to under 50 MB.

Alpine Linux does make it more difficult to install NPM modules with native dependencies, due to missing common libraries and tools needed to build them.

Therefore, we’re publishing the Alpine Linux image as a seperate tag (slim), rather than using this base image throughout our Dockerfiles.

This version should provide an extremely lightweight Node-RED image that works for most users.

1
docker run -it -p 1880:1880 nodered/node-red-docker:slim

Raspberry Pi Image

Node-RED is an incredibly popular tool for hacking on the Raspberry Pi. Using a custom Raspberry Pi image, developers can also have a full Docker system running in the Linux environment on their device.

So, can we use Docker to start Node-RED on the Raspberry Pi?

Due to the platform architecture, ARM rather than x86/x64 by Intel or AMD, Docker images must be packaged specifically for that platform. The existing Docker images created for Node-RED will not work.

Fortunately, there’s an existing RPi-compatible Docker image with Node.js and NPM.

Using this base image to build a new Raspberry Pi-specific Node-RED image, published with the rpi tag, means users can now start Node-RED on the Raspberry Pi using Docker.

1
docker run -it -p 1880:1880 nodered/node-red-docker:rpi

Serverless Go Actions

OpenWhisk, the open-source serverless platform, provides the ability to invoke custom Docker containers as serverless functions.

Developers can create new Actions, referencing public images on Dockerhub. OpenWhisk manages creating and executing containers using these images per invocation request.

Using this feature, developers can write serverless functions using the Go language. Compiled Go language binaries are embedded within custom Docker images and pushed into the platform.

So, how do we start?

This blog post will explain how to get your Go language functions running as “serverless functions” on OpenWhisk. If you’re impatient to get to the code, this repository contains the examples for everything discussed below.

OpenWhisk helps developers create custom Actions using Docker through an SDK…

OpenWhisk Docker SDK

Using the wsk command-line utility, developers can install the SDK into the current directory.

1
$ wsk sdk install docker

The SDK provides the source for a custom Docker image, which executes a custom binary in response to invocation requests. The default SDK copies the executable file, located at the client/action, into the image during the build process. Users build the image locally before pushing this to Dockerhub.

1
2
$ docker build -t <dockerhub_user>/docker_action .
$ docker push <dockerhub_user>/docker_action

Using the command-line utility, users can then create a new Action referencing this public Docker image. When this Action is invoked, the platform will spin up a new container from this custom image.

1
2
$ wsk action create docker_action --docker <dockerhub_user>/docker_action
$ wsk action invoke --blocking --result docker_action

OpenWhisk Docker Action

OpenWhisk SDK’s Docker image uses a Node.js application to handle the JSON invocation request from the platform and spawns a process to execute the binary. Invocation parameters are passed as a JSON string through a command-line argument to the binary. The executable must write the JSON response to stdout, the handler will return this to the platform.

Containers used to run OpenWhisk Actions must be expose a HTTP API on port 8080 with two paths, /init and /run. The platform sends HTTP POST requests to these paths to initialise the Action and schedule invocations.

The /init path is used to provide the Action source for languages which support runtime evaluation. User-provided Docker images do not need to implement this method, other than returning a non-error HTTP response.

The /run path is called by the platform for each invocation request. Parameters for the invocation are passed as the value property of the JSON request body. Any non-empty JSON response will be interpreted as the invocation result.

Go Actions using the Docker SDK

Using Go binaries with the Docker SDK requires the developer to cross-compile the source for the platform architecture and copy the binary to the client/action path.

1
2
3
4
export GOARCH=386
export GOOS=linux
go build -o action
mv action client/action

The Go code must parse the invocation parameters as a JSON string from the command-line argument. Data written to stdout will be parsed as JSON and returned as the Action response.

This sample Go source demonstrates using this method to implement a “reverse string” Action.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
package main

import "os"
import "encoding/json"
import "log"

type Params struct {
  Payload string `json:"payload"`
}

type Result struct {
  Reversed string `json:"reversed"`
}

// extract invocation parameters, passed as JSON string argument on command-line.
func params() Params {
  var params Params
  source := os.Args[1]
  buf := []byte(source)
  if err := json.Unmarshal(buf, &params); err != nil {
    log.Fatal(err)
  }
  return params
}

// convert struct back to JSON for response
func return_result(result Result) {
  buf, err := json.Marshal(result)
  if err != nil {
    log.Fatal(err)
  }
  os.Stdout.Write(buf)
}

func main() {
  input := params()

  // reverse the string passed from invocation parameters
  chars := []rune(input.Payload)
  for i, j := 0, len(chars)-1; i < j; i, j = i+1, j-1 {
    chars[i], chars[j] = chars[j], chars[i]
  }
  result := Result{
    Reversed: string(chars),
  }

  return_result(result)
}

Docker SDK Base Image

Building a base image from the OpenWhisk Docker SDK and publishing on Dockerhub simplifies the process of building a Docker-based Action. Developers can now use the following image (jamesthomas/openwhisk_docker_action), without having to install the SDK locally.

1
2
FROM jamesthomas/openwhisk_docker_action
COPY action /blackbox/action

This base image includes the Node.js handler to manage the platform HTTP requests. An executable file at /blackbox/action will be called for each invocation. JSON parameters and responses are still passed using command-line arguments and stdout.

Custom Go Handler

Using the Docker SDK for OpenWhisk relies on a Node.js application to handle the platform HTTP requests, spawning a process to execute the user binary file.

Implementing the HTTP API, described above, in Go would allow us to remove the Node.js handler from the image. Compiling the Go Action source with the HTTP API handler into a single binary and using an Alpine Linux base image will dramatically reduce the image size.

This should improve execution performance, by removing the Node.js VM process, and cold start-up time, through having a smaller Docker image.

Using this Go package, jthomas/ow, users can automate the process of creating Go-based Actions.

1
go get jthomas/ow

The package provides a method for registering Action callbacks and implements the HTTP endpoints for handling platform requests.

Invocation parameters are passed using a function parameter, rather than a raw JSON string. Returned interface values will be automatically serialised to JSON as the Action response.

1
2
3
openwhisk.RegisterAction(func(value json.RawMessage) (interface{}, error) {
   ...
}

Re-writing the “reverse string” Action above to use this package is shown here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
package main

import (
    "encoding/json"
    "github.com/jthomas/ow"
)

type Params struct {
    Payload string `json:"payload"`
}

type Result struct {
    Reversed string `json:"reversed"`
}

func reverse_string(to_reverse string) string {
    chars := []rune(to_reverse)
    for i, j := 0, len(chars)-1; i < j; i, j = i+1, j-1 {
        chars[i], chars[j] = chars[j], chars[i]
    }
    return string(chars)
}

func main() {
    ow.RegisterAction(func(value json.RawMessage) (interface{}, error) {
        var params Params
        err := json.Unmarshal(value, &params)
        if err != nil {
            return nil, err
        }
        return Result{Reversed: reverse_string(params.Payload)}, nil
    })
}

Cross-compiling the Action source, bundling this package, creates a single lightweight binary.

Embedding this file within a Docker image, using a minimal base image, creates a tiny image (<10MB). Containers from these images only execute a single process to handle both the HTTP requests and running the Action source.

1
2
3
4
FROM alpine:3.4
COPY action /action
EXPOSE 8080
CMD ["./action"]

Pushing the local image to Dockerhub and then using it to create an Action follows the same instructions above.

Conclusion

Running OpenWhisk Actions from user-provided Docker images allows developers to execute “serverless functions” using any language. This is a fantastic feature not currently supported by many of the other serverless providers.

OpenWhisk provides an SDK letting users build a local Docker image which executes their Action and handles the HTTP requests from the platform. Using this with Go-based Actions requires us to cross-compile our binary for the platform and handle passing JSON through command-line arguments and stdout.

Re-writing the HTTP handler natively in Go means the Docker image can contain and execute a single binary for both tasks. Using this Go package provides an interface for registering Actions and handles the HTTP requests automatically.

This project contains examples for the “reverse string” Action using both the Docker SDK and Go-based handler detailed above.

OpenWhisk and MQTT

OpenWhisk Feeds provide the mechanism to bind external events sources to serverless function executions.

Want to use OpenWhisk to listen for database updates to generate usage statistics? Or write Slack bots that respond to trigger words? Or notify users when Github project changes?

Rather than manually listening for these events with an external application and calling OpenWhisk Actions through the API, OpenWhisk Feeds automate connecting external events sources with Actions.

Feeds allow users to register Triggers to be invoked when external events occur. Defining Rules that bind these Triggers to Actions, we can have Actions run for external events.

OpenWhisk Packages

Feeds are contained within and accessible through Packages.

OpenWhisk provides numerous built-in packages under the whisk.system namespace. These packages contain both public Actions and Feeds.

1
2
3
4
5
6
7
8
9
10
11
$ wsk package list /whisk.system
packages
/whisk.system/alarms                                              shared
/whisk.system/cloudant                                            shared
/whisk.system/watson                                              shared
/whisk.system/system                                              shared
/whisk.system/weather                                             shared
/whisk.system/util                                                shared
/whisk.system/slack                                               shared
/whisk.system/samples                                             shared
/whisk.system/github

Retrieving the package summary, the Actions and Feeds contained within are returned. Feeds are referenced by the publisher’s namespace, package and feed name, e.g. /whisk.system/alarms/alarm

1
2
3
4
$ wsk package get /whisk.system/alarms --summary
package /whisk.system/alarms: Alarms and periodic utility
   (params: cron trigger_payload)
 feed   /whisk.system/alarms/alarm: Fire trigger when alarm occurs

The Alarm package (/whisk.system/alarms) contains a single Feed (/whisk.system/alarms/alarm) that calls the registered Trigger on a fixed schedule. Users provide the timer schedule through the cron parameter.

When creating new Triggers, users can specify a Feed source to bind their new Trigger to the external event source. Parameters from the command-line will be passed to the Feed source. The Feed provider will execute the Trigger each time an external event occurs.

1
$ wsk trigger create everySecond --feed /whisk.system/alarms/alarm -p cron '*/1 * * * * *' -p trigger_payload '{"vote":"Bernie"}'

This new Trigger will be invoked every second with the payload contents by the alarm Feed. Using rules, the Feed Trigger can be bound to call an Action on each invocation.

1
$ wsk rule create --enable alarmRule everySecond actionName

As well as using the built-in Feeds, users can create and register custom Feeds. This provides a way to integrate almost any external event source into the platform. Publishing custom Feeds within a public package will also make this event source available to all users on the systen, provided they know the package identifier.

Creating Custom Feeds

Users register new Feeds by providing a custom Action to the platform. This Action is invoked each time the Feed is bound to a new Trigger. Authentication credentials, supporting Trigger invocation through the OpenWhisk API, are passed in as invocation parameters.

This sample Action contains an outline for processing requests.

Feed Action
1
2
3
4
5
6
7
8
9
10
11
12
13
// params.lifeCycleEvent - Enum value (CREATE|DELETE) 
// params.triggerName - User's Trigger identifier to call
// params.authKey - Authentication details for calling Trigger

function main (params) {
  if (params.lifecycleEvent === 'CREATE') {
    create(params);
  } else if (params.lifecycleEvent === 'DELETE') {
    remove(params)
  }

  return whisk.async();
}

The params argument contains the Trigger information provided by the platform and any parameters from the user during creation.

The lifeCycleEvent parameter is a string value, informing the Feed provider whether to register (CREATE) or remove (DELETE) the user’s Trigger with the event source. The Trigger identifier is passed as the triggerName parameter, with the authentication key (authKey) used for the API requests.

Feed Actions must be registered with a custom annotation (feed), allowing the platform to distinguish them from “normal” Actions. This annotation can be set during the create command.

1
$ wsk action create -a feed true feed_name feed_action.js

Once a custom Feed Action has been registered, users can create new Triggers using that Feed source, following the steps above.

MQTT Feeds

The “Internet of Things” is often cited as a common usecase for serverless platforms. Solutions are often event-driven and stateless, e.g. wait for data from this device, do some processing and then store the results in this database.

MQTT is a lightweight publish-subscribe messaging protocol, commonly used for edge of network device-to-device communication.

Bridging MQTT messages to OpenWhisk Actions can be achieved by creating a new Feed provider. This provider would subscribe to message topics and execute registered Triggers with incoming messages.

The custom feed provider would need to establish and maintain long-lived MQTT connections, waiting for messages to arrive. This requirements means the Feed provider needs an external service to handle managing these connections, it won’t be possible within the Feed Action.

This feed provider service is implemented using Node.js, using Cloudant for the database. The service listens for HTTP requests, with Trigger registration details, from the Feed Action. The Node.js MQTT library is used to subscribe to registered topics. When messages are received, the OpenWhisk client library is used to invoke the Trigger remotely, passing the message contents as event parameters.

This service provider is packaged using Docker.

Pushing this image into the IBM Containers registry, the Feed provider can be started on IBM Bluemix using the Containers service.

Pushing feed provider to IBM Containers
1
2
3
$ docker build -t USERNAME/mqtt_feed_provider .
$ docker tag USERNAME/mqtt_feed_provider registry.ng.bluemix.net/USERNAME/mqtt_feed_provider
$ docker push registry.ng.bluemix.net/USERNAME/mqtt_feed_provider

Registering Feeds

With the Feed service provider running, the Feed Action can be deployed.

The Feed will be registered under the name, mqtt_feed_provider, in a custom package, mqtt.

Using the –shared command-line flag, the Feed package can be registered as a public package. Feeds and Actions within public packages are visible to every system user.

Rather than hardcoding the service provider location within the Feed Action, this configuration value will be accessible as a package parameter. This can be updated at runtime with modifying the Feed Action source.

1
2
$ wsk package create --shared -p provider_endpoint "http://CONTAINER_IP:3000/mqtt" mqtt
$ wsk package update mqtt -a description 'MQTT topic feed. Messages received on broker topic as passed to triggers"

Having created the package, we can add the Feed Action, using the custom attribute to denote this is a Feed Action.

1
$ wsk action create -a feed true mqtt/mqtt_feed mqtt_feed.js

Once the Feed has been registered, it can be referenced when creating new Triggers.

1
$ wsk trigger create feed_trigger --feed /james.thomas@uk.ibm.com_dev/mqtt/mqtt_feed -p topic 'whiskers' -p url 'mqtt://test.mosca.io'

MQTT broker url and topic name are passed as Trigger parameters, using the -p flags. These values are included within the invocation arguments to the Feed Action, shown below.

1
2
3
4
5
6
7
8
var params = {
  authKey: 'USERNAME:PASSWORD',
  url: 'mqtt://test.mosca.io',
  provider_endpoint: 'http://CONTAINER_IP:3000/mqtt',
  topic: 'whiskers',
  lifecycleEvent: 'CREATE',
  triggerName: '/james.thomas@uk.ibm.com_dev/feed_trigger'
}

Once the Feed service provider has connected to the broker and subscribed to the topic, incoming messages will register as Trigger events invocations for the public_feed Trigger.

Using this custom Feed, users can easily connect MQTT messages to OpenWhisk Actions.

Github Project

Source code for this custom OpenWhisk Feed is available here. The project contains the Feed Action and Provider service. The README contains the deployment and usage instructions.

Cognitive Bots With IBM Watson

Later this month, I’m speaking at Twilio’s conference about building cognitive bots with IBM Watson. Preparing for this presentation, I’ve been experimenting with the IBM Watson services to build sample bots that can understand, and act on, natural language.

IBM’s artificial intelligence system, Watson, now provides a series of “cognitive” services available through IBM’s Bluemix cloud platform. Developers can integrate everything from natural language processing, image and speech recognition, emotion analysis and more into their applications using RESTful APIs.

The Watson Developer Cloud site has numerous sample apps to help you understand how to integrate the services together to build “cognitive” bots.

In one of the samples, the Dialog service is used to develop a pizza ordering bot. Users can order a pizza, specifying the size, toppings and delivery method, using natural language.

After understanding how this sample worked, I had an idea to enhance it with the tone analysis service

Where the heck is my pizza?

Let’s imagine the customer has ordered a delivery using pizza-bot and the driver is being (even) slower than normal.

If the customer asks

“Where is my pizza?”

We return the standard message all pizza takeaways use when calling to inquire where the driver is….

“The driver has just left, he’ll be ten minutes.”

An hour later…

When the driver still hasn’t arrived, the customer would probably ask again and with a bit less civility…

“Where the heck is my pizza? I ordered an hour ago! This is ridiculous.”

At this point, the “just ten minutes” reply is not going to be well received!

Building bots that can understand conversation tone will mean we can script a suitable response, rather than infuriating our hungry customers.

Using the tone analyser service, I wanted to enhance the sample to use conversation sentiment to affect the dialogue. Bot responses should be generated based upon both user input and conversation sentiment.

Let’s review both services before looking at how to combine them to create the improved pizza bot…

IBM Watson Dialog

The IBM Watson Dialog service enables a developer to automate scripting conversations, using natural language, between a virtual agent and a user. Developers build up a decision tree for dialogue, using a markup language to define the conversation paths.

Developers can then utilise the pre-defined linguistic model to converse with users. The system will keep track of the conversation state when processing user input to generate a suitable response. It can also store conversation properties, either extracted from user input or manually updated through the API.

These conversation properties can be used to control the dialogue branching.

Documentation on the service is available here.

IBM Watson Tone Analyser

The IBM Watson Tone Analyzer Service uses linguistic analysis to detect three types of tones from text: emotion, social tendencies, and language style.

Emotions identified include things like anger, fear, joy, sadness, and disgust. Identified social tendencies include things from the Big Five personality traits used by some psychologists. These include openness, conscientiousness, extroversion, agreeableness, and emotional range. Identified language styles include confident, analytical, and tentative.

Documentation on the service is available here.

Extending Pizza Bot

Enhancing pizza bot to support dialogue about delivery times, we can start by identifying when the user is asking about the pizza delivery. At this point, unless the user is angry, we can return the default response. When sentiment analysis indicates this user is angry, we should branch to returning a more sympathetic message.

Matching User Input

Matching user input about delivery times, there a few common questions we want to capture.

  • Where’s my order?
  • How long will it be until my pizza arrives?
  • When will my takeout get here?

Creating our new conversation branch within a folder element will allow us to group the necessary input, grammar and output elements as a logical section.

Order Querieslink
1
2
3
4
5
6
7
8
9
10
11
12
<folder label="Order">
  <input>
    <grammar>
      ...
    </grammar>
    <output>
      <prompt selectionType="RANDOM">
        ...
      </prompt>
    </output>
  </input>
</folder>

This structure will process the output element, to generate the bot reply, only if the input grammar matches user input. Adding item nodes under the input’s grammar element will let us define the dialogue matching criteria, shown here.

Query Grammarlink
1
2
3
4
5
6
7
8
<grammar>
  <item>$where* order</item>
  <item>$where* pizza</item>
  <item>$how long* order</item>
  <item>$how long* pizza</item>
  <item>$when * order * here</item>
  <item>$when * pizza * here</item>
</grammar>

Using wildcard matching characters, $ and *, means the grammar (“$where * order”) will match questions including “Where is my pizza?” and “Where’s my pizza?” rather than having to manually define every permutation.

People often use synonyms in natural language. Rather than manually defining grammar rules for all alternative words for pizza and order, we can add concept elements to automatically match these. The sample already has a concept element defined for the pizza term, we only have to add elements for order.

Concept Entitieslink
1
2
3
4
5
6
7
8
<concept>
  <grammar>
    <item>Order</item>
    <item>Takeaway</item>
    <item>Takeout</item>
    <item>Delivery</item>
  </grammar>
</concept>

Grammar rules which include the order term which automatically match takeaway, takeout or delivery.

Adding Default Response

Having matched the user input, we want to return the default response from a pre-specified list.

Bot Replieslink
1
2
3
4
5
6
7
8
<output>
  <prompt selectionType="RANDOM">
    <item>I've just checked and the driver is ten minutes away, is there anything else I can help with?</item>
    <item>Hmmm the driver's running a bit late, they'll be about ten minutes. Is there anything else I can help with?</item>
    <item>They should be with you in ten minutes. Is there anything else I can help with?</item>
  </prompt>
  <goto ref="getUserInput_2442994"/>
</output>

Handling Angry Customers

Within the dialog markup, profile variables can be defined to store conversation entities. These variables can be referenced by conditional branches in the markup to control responses.

Defining a new profile variable for the anger score, this value can be updated manually before the current user input is processed to return the dialogue response.

Profile Variablelink
1
2
3
4
5
6
<variables>
  <var_folder name="Home">
    ...
    <var name="anger" type="NUMBER" initValue="0" description="Anger emotion score for conversation."/>
  </var_folder>
</variables>

Adding a child branch, for the conditional response, after the input grammar will allow us to return a custom response if the profile variable for the anger emotion is above a threshold.

Anger Branchinglink
1
2
3
4
5
6
7
8
9
10
11
12
13
<folder label="Order">
  <input>
    <grammar>
      <item>$where* order</item>
    </grammar>
    <if matchType="ANY">
      <cond varName="anger" operator="GREATER_THEN">0.50</cond>
      <output>
        <prompt selectionType="RANDOM">
          <item>Please accept our apologies for the delivery driver being very late. Could you call us on 0800 800 800 and we'll get this fixed?</item>
        </prompt>
      </output>
    </if>

When we’ve detected the user is angry about the delivery delay, we direct them to ring the restaurant to find out what’s happened to the driver.

Combining Watson Services

Modifying the backend service that calls the Watson services, we’re now passing the user’s input through the Tone Analyzer service and manually updating user’s anger score in their profile, before calling the Dialog service.

This anger score will be used to control the dialogue response in real-time.

Using Tone Analyserlink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
app.post('/conversation', function(req, res, next) {
  tone_analyzer.tone({ text: req.body.input }, function(err, tone) {
    var categories = tone.document_tone.tone_categories
    var emotion_tones = categories.find(function (tone) {
      return tone.category_id === 'emotion_tone'
    })

    var anger_tone = emotion_tones.tones.find(function (tone) {
      return tone.tone_id === 'anger'
    })

    var params = {client_id: req.body.client_id, dialog_id: dialog_id, name_values: [{name: 'anger', value: anger_tone.score}]}
    dialog.updateProfile(params, function (err, results) {
      var params = extend({ dialog_id: dialog_id }, req.body);
      dialog.conversation(params, function(err, results) {
        else
          res.json({ dialog_id: dialog_id, conversation: results});
      });
    })
  });
});

The commit log for the fork shows the full changes needed to integrate this feature.

Conclusion

Bots are a huge trend for 2016. One of the major challenges to developing your own bots is handling user input using natural language. How can you go beyond simple keyword matching and regular expressions to build solutions that actually understand what your user is asking?

Using the IBM Watson Dialog service users can script natural language conversations. Defining a linguistic model for their dialogue using markup language, the system can use this to process natural language and return the appropriate response. Conversation entities are recognised and stored in a user profile.

Combining this service with the IBM Watson Tone Analyzer, users can script conversations that use the user’s emotional tone to modify the response.

Modifying the pizza sample, we incorporate the anger score to return a more appropriate response when the user is angry about their delivery being delayed.

IBM Watson has many other services that can be integrated with the Dialog service using the same pattern to build “cognitive” bots. Using these services takes the hard work out of building bots that actually understand and respond with emotion to input using natural language.

Serverless APIs With OpenWhisk and API Connect

“Serverless” cloud platforms are a major trend in 2016. Following on from Amazon’s Lambda service, released eighteen months ago, this year has seen IBM, Microsoft and Google all launch their own solutions.

These platforms let you build stateless microservices, combining APIs with business logic, without servers. Microservices are executed on-demand, in milliseconds, rather than having to sit idle waiting for incoming requests. Users pay only for the raw computation time used.

Combining serverless APIs with static file hosting for site resources, e.g. HTML, JavaScript and CSS, means we can build entire serverless web applications.

Playing with OpenWhisk recently to build simple microservices, I began to investigate using the platform to build the APIs for serverless applications.

How can we use OpenWhisk to define a new microservice and then expose that service as an API with a HTTP interface?

Let’s start by looking at OpenWhisk…

OpenWhisk

Using the OpenWhisk platform, developers register small bits of code, known as Actions, that can be invoked on-demand. These functions can be written in Node.js, Swift or Docker images. Let’s look at a simple Node.js Action that takes a parameter and returns a message with that value.

OpenWhisk Action
1
2
3
4
5
function main(params) {
  return {
    payload: 'Hello ' + params.name
  };
}

Node.js actions must include a function named main. OpenWhisk executes this function for each invocation, passing request parameters as arguments. Return values from the function will be included in the response.

Using the OpenWhisk command-line utility, we turn this local JavaScript code into a remote action.

1
2
3
4
5
6
7
[~/code/serverless]$ ls
source.js
[~/code/serverless]$ wsk action create hello_action source.js
ok: created action hello_action
[~/code/serverless]$ wsk action list
actions
/james.thomas@uk.ibm.com_dev/hello_action                         private

With the action registered, we can test the service from the command-line.

1
2
3
4
5
6
7
8
9
10
[~/code/serverless]$ wsk action invoke -b hello_action -p name "Bernie Sanders"
ok: invoked hello_action with id 429b35c3e3ac494ea902390ca64afe32
response:
{
    "result": {
        "payload": "Hello Bernie Sanders"
    },
    "status": "success",
    "success": true
}

We can also update the action to use default parameter values.

1
2
3
4
5
6
7
8
9
10
11
12
13
[~/code/serverless]$ wsk action update hello_action -p name "Donald Trump"
ok: updated action hello_action
[~/code/serverless]$ wsk action invoke -b hello_action
ok: invoked hello_action with id 0299bf2baf9242b7a00a8095caaeb7a4
response:
{
    "result": {
        "payload": "Hello Donald Trump"
    },
    "status": "success",
    "success": true
}
[~/code/serverless]$

Registered actions can be executed manually, using an authenticated API request, or automatically, hooking actions to triggers and feeds using rules. For more details on triggers, feeds and rules, please see the OpenWhisk documentation.

The command-line utility translates commands into HTTP requests to the OpenWhisk API.

Pro-Tip: Adding the ‘-v’ flag when using command-line utility will show HTTP traffic sent to the OpenWhisk API.

Serverless APIs With OpenWhisk

Building backend services for serverless web applications, there were two challenges to resolve before invoking these APIs from client-side JavaScript code.

  • Authentication. OpenWhisk API requests require HTTP authentication, using the developer’s credentials. Embedding these credentials within client-side files is a terrible idea…

  • Cross-Domain Requests. CORS support is not enabled on the OpenWhisk platform. Calling services from a browser would mandate us having CNAME records configured with an external domain.

Authentication needs to be resolved, while cross-domain support is an inconvenience.

Using OpenWhisk on IBM Bluemix, we have access to a huge range of cloud services to help build applications. Reviewing the catalogue, there’s a new service API Connect which can help us resolve both issues with minimal effort.

API Connect

Announced in February, API Connect is IBM’s new “API Management-as-a-Service” solution. Developers can use the service for creating, running, managing and securing APIs in the cloud.

Using this service, we can construct new public APIs, with CORS support, that proxy the authenticated OpenWhisk APIs used to trigger our services. Using these APIs from our serverless frontends will be possible without leaking credentials or having to configure DNS records.

Once we’ve signed up for an account with API Connect, you need to install the developer toolbox locally. Using this tool will allow us to construct new APIs and publish them to the cloud.

TLDR: I’ve exported the sample flow configuration generated below here. Import this YAML file into the API Connect editor, replacing USERNAME, PASSWORD and NAMESPACE, before deploying this flow to IBM Bluemix.

API Editor

Install the API Connect Toolkit using NPM and run the following command to open the editor.

1
2
$ npm install -g apiconnect
$ apic edit

Using the APIs panel, select the Add button. Provide a title for your service.

Leave the Add to a new product checkbox selected and provide a title for the product.

The editor now shows the Design panel, allowing you to define the external public API schema.

We’re going to define a single endpoint (/hello-name) which supports HTTP GET requests with a single query parameter.

Adding the endpoint

Disable the clientID definition under the “Security” panel and then scroll down to the Paths section.

Add a new path for the endpoint /hello-name. Set a parameter for this path, using the identifier name from location as query and type as string.

Move to the Definitions section to define the API response schema. We want to return a JSON object with a single property, result, that contains the JSON object returned from the Action response.

Add a new Definition, named whisk_response and type as object, with a single object property, result.

Under the Paths panel, expand the GET operation. Set the schema for the 200 response to whisk_response.

CORS supported is already enabled by default (under the Lifecycle section). Click the Save icon in toolbar and then move to the “Assemble” tab.

Defining API operations

Having defined the public API schema, we need to implement the API operations.

On the “Assemble” tab, the flow editor allows us to connect different backend operations to construct our service. IBM Bluemix only supports deploying flows constructed with the “DataPower Gateway policies” nodes. Microgateway nodes, e.g. Javascript, are not supported.

Invoking OpenWhisk Actions

The default flow contains a single invoke node. This node type makes HTTP requests, passing the result to the next node in the flow.

Use this node to execute your OpenWhisk Action by bringing up the editor and changing the URL to the correct endpoint, e.g. https://openwhisk.ng.bluemix.net/api/v1/namespaces/YOUR_NAMESPACE/actions/ACTION_ID?blocking=true

Make sure to include the query parameter, blocking=true. This makes OpenWhisk wait until the Action has completed execution before returning, rather than after invocation starts.

Change the HTTP method from GET to POST and fill in the username and passwords fields.

Add the value invoke_result to the Response Object Variable field. This will save the HTTP response into a context variable we can reference in the following map node definition.

Passing Query Parameters

Invoking OpenWhisk Actions through the API uses a HTTP POST request, passing parameters within the JSON body. Our external API supports HTTP GET operations, with parameters through query string values in the URL.

Using the map node in the flow will translate between these two methods.

Drag a map node from the left-hand panel and drop it on the wire between the circle and the invoke node.

Open the map node editor and add a new input parameter. Change the context variable to request.parameters.name with type string. This contains the query parameter value we’re using to pass in action arguments.

Returning to the map node editor, add a new output parameter. Leave the Context variable as message.body. This variable will be used by the invoke node to populate the request body.

Change the Content Type to application/json. Select the definition as inline schema to define the JSON schema for the HTTP POST body. Add the following JSON Schema definition to the editor form.

JSON Schema
1
2
3
4
5
6
7
8
{
  "properties": {
    "name": {
      "type": "string"
    }
  },
  "type": "object"
}

With the input and output formats defined, we can wire the two parameters together. Under the Map panel, click the dot next to the input parameter and then click the second dot on the right, next to the name:string label.

Remember to click Save before proceeding.

Returning Action Result

OpenWhisk Action API invocation responses include both the action result payload and meta-data about the invocation event.

Sample Invocation Event
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
{
  "name": "hello_action",
  "subject": "james.thomas@uk.ibm.com",
  "activationId": "5388b29e9f134737baf57bd12257dfd7",
  "publish": false,
  "annotations": [],
  "version": "0.0.1",
  "response": {
    "result": {
      "payload": "Hello Bernie"
    },
    "success": true,
    "status": "success"
  },
  "end": 1461667635975,
  "logs": [],
  "start": 1461667635970,
  "namespace": "james.thomas@uk.ibm.com"
}

Rather than returning the raw result, we only want to return the result payload property (response.result). Using another map node we can define a subset of the invoked API response to be the HTTP response body.

Add a second map node to the flow, this time after the invoke node.

Add a new input property. We previously set a context variable in the invoke definition that will contain the API response (invoke_api). The response body is available as the body property of this variable.

Edit the context variable to be invoke_api.body.response.result to set the input property as the child property of the invoke result. Set the content type to application/json and schema to object.

Add a new output property. Leave the context variable as message.body. This context variable is used as the response body.

Set content type to application/json and change the definition to #/definitions/whisk_response. This was the JSON schema we created during the external API definition.

Returning to the map overview, wire together the input property to the result attribute of the output property.

Click the Save icon before making any further changes.

Using the invoke and map nodes, we’ve now implemented our external API. Making our API live requires us to deploy the flow definition to IBM Bluemix.

Deploying to IBM Bluemix

After saving your flow, click the Publish icon in the top-right hand corner. We’re going to publish to the default Sandbox target. Follow the steps to find and add this target to the local editor.

Once you’ve added Sandbox as the default target, select Publish and click the configured catalogue. On the dialog box, select the Select Specific Products option and choose the openwhisk product.

Clicking the confirmation button will upload our API definition to the external API Connect platform.

If everything has been configured and deploying correctly, your new API should now be live!

Let’s test it…

Testing

Opening the API Connect dashboard, the sandbox catalogue should now contain the openwhisk product with the public API we defined using the editor.

We can now verify this API works by making the HTTP request to the endpoint. Under the Settings tab, the API Endpoint section contains the Base URL for our API catalogue. APIs deployed under this catalogue will use this path as the endpoint root.

The API definition registered a relative URL path, /hello-name, which we can combine with the catalogue endpoint (e.g. https://api.us.apiconnect.ibmcloud.com/USER_ORG_SPACE/sb) to generate an public API endpoint.

We can now test this API by sending a HTTP GET request to the URL, passing the name as a query parameter.

1
2
3
4
5
6
7
8
9
10
11
12
13
[17:13:10 ~]$ http get https://api.us.apiconnect.ibmcloud.com/jamesthomasukibmcom-dev2/sb/hello-name?name="Bernie Sanders"
HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Type: application/json
Date: Tue, 26 Apr 2016 16:24:36 GMT

{
    "result": {
        "payload": "Hello Bernie Sanders"
    }
}

[17:24:36 ~]$

It works! 😃

We’ve successfully used API Connect to create an external API which proxies the OpenWhisk API. We now have a public endpoint we can use to invoke OpenWhisk Actions, without exposing our credentials and enabling CORS-support for cross-domain XHRs.

Conclusion

Serverless computing platforms give developers a rapid way to build APIs without servers. Combining this approach for building backend services with static file hosting provides an architecture for developing entire serverless web applications.

Experimenting with OpenWhisk as the backend platform for building serverless web applications, there were two challenges, authentication and cross-domain support.

Both issues were resolved using the API Connect service on IBM Bluemix.

API Connect is an incredibly powerful tool for creating, running, managing and securing APIs. Using the editor application to construct a new API, the endpoint was implemented using the invoke and map nodes. Deploying the generated flow to IBM Bluemix exposed the API as a public endpoint.

Playing With OpenWhisk

IBM recently launched OpenWhisk, their new “serverless” compute platform.

This service allows developers to register small bits of code that are executed on-demand in response to external events. The “serverless” stack started in 2014, when Amazon launched Lambda, but is now set to be a major technology trend in 2016 with IBM, Microsoft and Google all launching their own solutions.

OpenWhisk is the first open-source “serverless” platform. It supports running registered actions in Node.js, Swift and even executing custom Docker containers.

Playing around with the technology recently, I’ve created two projects using the platform.

OpenWhisk Client Library

OpenWhisk exposes a RESTful API for interacting with the service. Wrapping this API with a small client library makes it easy for developers to interact with the service from JavaScript.

This library has been donated back to the OpenWhisk project and is available through NPM.

1
2
3
4
5
const openwhisk = require('openwhisk')
const ow = openwhisk({api: 'https://openwhisk.ng.bluemix.net/api/v1/', api_key: '...', namespace: '...'})
ow.actions.invoke({actionName: 'action'}).then(result => {
  // result is service response
})

Whiskify

This project, available through NPM, makes it easy to run arbitary JavaScript functions as OpenWhisk actions. Passing a reference to a JavaScript function into the module, an OpenWhisk action is created using the function source. The module returns a new JavaScript function, that when executed, will call the remote action and returns a Promise with the service response.

1
2
3
4
5
6
7
8
9
10
11
12
const whiskify = require('whiskify')({api: 'https://', api_key: '...', namespace: '...'})
const action = whiskify(function (item) { return item + 1; })

action(1).then(function (result) {
  // == 2
})

action.map([1, 2, 3, 4]).then(function (result) {
 // == [2, 3, 4, 5]
})

action.delete()

This project uses the client library above.

Debugging Live Containers on IBM Bluemix

For the last few months, I’ve been using the ELK stack to collect logs from my Cloud Foundry applications. This service has been deployed on IBM Bluemix using a Docker container, previously detailed in this blog post, and running happily until it ran into issues this week.

Trying to load the Kibana web application, the server was returning connection refused errors. Looking at the container in the IBM Bluemix dashboard showed no obvious signs of issues. Reviewing the container log output uncovered nothing indicating what had failed.

Hmmm…

Fixing this issue would require me to start debugging from within the live container, but how?

This container image had not included an SSH daemon that would allow remote access over SSH.

Looking over the documentation for the IBM Containers plugin for the Cloud Foundry CLI, I noticed the exec command.

Docker exec allows a user to spawn a process inside their Docker container via
the Docker API and CLI.

Since Docker 1.3, released in October 2014, the exec command has allowed users to run new commands within existing containers.

The IBM Containers implementation now supports this Docker command.

Using the IBM Containers plugin for the Cloud Foundry CLI, I can find the container id for the instance I want to debug and then start a bash shell to start resolving my issue.

1
2
$ cf ic ps
$ cf ic exec -it <container_id> /bin/bash

Having a live shell to my container allowed me to resolve the issue within a few minutes, without having to affect the running state of the container. This command also removes the need to keep an SSH daemon running on containers for remote access.

For more information on the subset of Docker commands supported by IBM Containers, see the following documentation.

Cloud Foundry Application Monitoring Bot for Slack

Cloud Foundry makes it so easy to build, deploy and manage applications that it can be a struggle just to keep up with development progress…

“Who is restarting this application?”
“What is this new service instance?”
“When did this application instance run out of memory?”

Development teams are increasingly using Slack to collaborate on projects and using custom bots to manage and monitor applications, triggered through the channel messages. This approach, popularised by Github, has now become known as “ChatOps”. Using group chat for development projects gives greater operational visibility to everyone in the team.

Slack has exploded in use over the past two years, recently signing up more than a million active users. The platform publishes an API for writing bots that respond automatically to messages, allowing users to write custom integrations for external services.

Users can register webhooks to receive channel messages, based upon keyword triggers, and allow bots to reply with new channel messages. The platform also provides a websocket channel with registered bots for real-time communication.

Could we write a custom bot for monitoring applications on the Cloud Foundry platform?

The bot would publish notifications about applications and services into group channels, helping keep teams updated with platform events in real-time.

Cloud Foundry Monitoring APIs

Cloud Foundry provides access to the platform through a series of RESTful APIs, exposed by the Cloud Controller component. User commands from the CF CLI tool are translated into calls to these APIs.

Tip: Setting the CF_TRACE environment parameter to true will show the API calls generated by the CLI commands.

Platform user account credentials are used to obtain OAuth2 tokens for authenticating service calls.

Looking at the documentation, there’s an endpoint for retrieving all platform events. This API is used to retrieve events for an application when using the CF CLI events command. Events can be filtered by the application, event type and timestamps. Responses include events about changes to applications, services and service instances.

Polling this API, with timestamp filtering to ignore old events, we can retrieve a continuous stream of new platform events.

Slack Integration

Setting up a new bot integration for a Slack group provides you with a token you can use to authenticate with the Real-Time Messaging API. Rather than having to implement the Websocket-based API handler ourselves, we can use one of the many existing community libraries.

Using the Node.js client library, passing in the authentication token, we just need to implement callback handlers for the API events.

Slack Client
1
2
3
4
5
6
7
8
9
10
11
12
var Slack = require('slack-client')

var slackToken = 'xoxb-YOUR-TOKEN-HERE' # Add a bot at https://my.slack.com/services/new/bot and copy the token here.
var autoReconnect = true # Automatically reconnect after an error response from Slack.
var autoMark = true # Automatically mark each message as read after it is processed.

var slack = new Slack(slackToken, autoReconnect, autoMark)
slack.on('message', function (message) {...})
slack.on('error', function (err) {...})
slack.on('open', function () {})

slack.login()

When platform events occur, we forward these to any channels the bot is registered in.

Plugging together the Cloud Foundry event monitoring code with the Slack bot integration, cfbot was born…

cfbot

This Cloud Foundry monitoring bot can be deployed to… Cloud Foundry!

You will need to register the bot with your Slack group to receive an authentication token. This token, along with login details for a platform account, need to be created as user-provided service credentials. The bot will read these service credentials on deployment and start monitoring for events.

Full installation instructions available in the project README.

usage

cfbot will monitor events from applications in all spaces and organisations that the user account has access to.

Users can filter the applications and events being reported using the apps and events commands. Both commands take application or event identifiers that are used to match incoming events. The wildcard ‘*’ identifier can be used to revert to matching all events.

@cf apps // show the currently application filter
@cf apps app_name // add the 'app_name' to the filter list
@cf apps * // reset to the filter to wildcard matching

@cf events // show the currently event filter
@cf events event_type // add the 'event_type' to the filter list
@cf events * // reset to the filter to wildcard matching

@cf status // show the current bot status message

@cf polling_frequency // show the cf events api polling time in seconds
@cf polling_frequency 10 // set the cf events api polling time in seconds

The following events are currently registered:

  • App Creation and Deletion Events.
  • App Lifecycle Events (start, stop, restart, restage)
  • Instance Crash Events.
  • Service Creation, Deleting and Binding.
  • Scaling (memory, CPU, disk)
  • Routes Changes (map, unmap)

Other bots

Other people have written Cloud Foundry bots before cfbot. Here are the other projects I discovered that might be useful…