Starting OpenWhisk In Sixty Seconds

Jan 19, 2018
openwhisk serverless docker
7 min read

Apache OpenWhisk is an open-source serverless platform. Developers can use hosted instances from IBM, deploy to any infrastructure provider and even run it locally.

Developers often use a local instance of the platform during development. Deploying to a local instance is faster than the cloud. It also provides access runtime environments to debug issues and allows development without an Internet connection. Production applications are still run on IBM Cloud Functions.

But OpenWhisk provides numerous options for starting the platform, including running the platform services directly, using container management tools like Kubernetes and Mesos or starting a pre-configured virtual machine with Vagrant.

So, what’s easiest?

OpenWhisk Devtools.

Using this project, the platform can be started on any machine with Docker Compose in around sixty seconds. Before we explain how this works, let’s show the steps needed to spin up the platform using the project.

openwhisk in around sixty seconds…

Do you have Docker with Compose support installed? If not, follow the instructions here.

Start the platform with the following commands.

$ git clone git@github.com:apache/incubator-openwhisk-devtools.git
$ cd incubator-openwhisk-devtools/docker-compose
$ make quick-start

Having cloned the repository, creating the local instance only takes around sixty seconds! 💯

$ time make quick-start &>/dev/null

real    1m10.128s
user    0m1.709s
sys     0m1.258s

Platform services will be running as containers on the host after initialisation.

$ docker ps --format "&#123;&#123;.ID&#125;&#125;: &#123;&#123;.Names&#125;&#125; &#123;&#123;.Image&#125;&#125;" 
17c5d31e2c20: wsk0_60_prewarm_nodejs6 		(openwhisk/nodejs6action:latest)
0eace484289c: wsk0_59_prewarm_nodejs6 		(openwhisk/nodejs6action:latest)
1be725d8767c: openwhisk_apigateway_1          (adobeapiplatform/apigateway:1.1.0)
641cbabeb790: openwhisk_kafka-topics-ui_1 	(landoop/kafka-topics-ui:0.9.3)
f52c25dbadd9: openwhisk_controller_1 		    (openwhisk/controller)
8f0c6aa14ccc: openwhisk_invoker_1 			(openwhisk/invoker)
d5274194f842: openwhisk_kafka-rest_1 		    (confluentinc/cp-kafka-rest:3.3.1)
40a1585f64bb: openwhisk_kafka_1 	            (wurstmeister/kafka:0.11.0.1)
b0b0f75c6fdb: openwhisk_db_1 			        (couchdb:1.6)
a7449c2edc4d: openwhisk_zookeeper_1 		    (zookeeper:3.4)
178abe09b793: openwhisk_redis_1 		        (redis:2.8)

…and that’s it!

testing it out

setting up CLI tool

OpenWhisk provides a CLI tool for interacting with the platform. The quick-start command automatically writes account credentials for the local instance into the CLI configuration file. Using the CLI tool to print current configuration values shows the platform endpoint set as the local machine ip or hostname.

If you don’t have the CLI tool already installed, the project downloads the binary to the following location: devtools/docker-compose/openwhisk-master/bin/wsk

$ wsk property get | grep host
whisk API host		localhost

The local instance is configured with a single user account (guest) with these credentials. Administrative credentials are stored in this configuration file.

creating sample actions

With the CLI configured correctly, you can create and invoke an action on the local platform instance.

Create a new file called hello.js with the following contents.

function main (params) {
  var name = params.name || 'World'
  return { payload: 'Hello, ' + name + '!' }
}

Create a new action called hello from the local hello.js file.

$ wsk action create hello hello.js
ok: created action hello

List the actions registered on the platform.

$ wsk action list
actions
/guest/hello                                                           private nodejs:6

Invoke the action, blocking until it has finished.

$ wsk action invoke -r hello -p name James
{
  "payload": "Hello, James!"
}

Retrieve the activation record.

$ wsk activation list
activations
1d16d13fdbba4fdc96d13fdbba7fdc76 hello
$ wsk activation get 1d16d13fdbba4fdc96d13fdbba7fdc76
ok: got activation 1d16d13fdbba4fdc96d13fdbba7fdc76
{
  "namespace": "guest",
  "name": "hello",
  "version": "0.0.1",
  "subject": "guest",
  "activationId": "1d16d13fdbba4fdc96d13fdbba7fdc76",
  "start": 1516364811767,
  "end": 1516364811769,
  "duration": 2,
  "response": {
    "status": "success",
    "statusCode": 0,
    "success": true,
    "result": {
      "payload": "Hello, James!"
    }
  },
  ...
}

how does it work?

Apache OpenWhisk is an open-source serverless platform, composed of microservices written in Scala. Custom runtime code is bundled and managed as platform services using Docker. The platform also uses numerous external open-source projects, including CouchDB, Kafka, Zookeeper and Redis. Existing public images are used to pull those dependencies into the project.

Apache OpenWhisk automatically builds and publishes images for custom runtime services on Docker Hub. This means containers can be started from public images, rather than having to build them manually. Using the existing images dramatically reduces the start-up time for getting the project running locally.

“Devtools” uses Docker Compose to start a local instance of the platform. Docker Compose coordinates starting containers in the correct order, injects runtime configuration properties and link services using local networking interfaces.

tips & tricks

Once you have a local instance of the platform running, there are a few things you can do to make it easier to use during development…

switching CLI between platform instances

Using a local instance of the platform for testing and development, you will still want to deploy production applications to IBM Cloud Functions. Switching between these platform instances relies on updating the CLI configuration file with the correct authorisation credentials each time.

Rather than manually updating the default configuration file each time, an environment variable (WSK_CONFIG_FILE) can be used to choose a different configuration file. Keep credentials for different instances in separate files. Use the environment parameter to quickly switch platform instances.

$ WSK_CONFIG_FILE=~/.wskprops-local wsk

Using a shell alias to expose a new command to do this automatically makes this even easier.

alias wsk-local='WSK_CONFIG_FILE=~/.wskprops-local wsk -i'

The -i flag is used because the local platform instance uses a self-signed SSL certificate.

view platform details in database

CouchDB is the platform data store. This stores all installed actions, save activation records and other runtime properties. Accessing this database can be helpful to diagnose issues directly.

CouchDB comes with a administration web application called “Futon”. This can read and modify database documents, run view queries and check configuration parameters.

Open this URL to access Futon for the local platform instance: http://localhost:5984/_utils/

Docker is configured to forward networking traffic from local port 5984 to the same port on the container.

read platform logs

System logs, generated from platform containers, are stored in the ~/tmp/openwhisk directory.

Logs from the invoker are stored in the invoker/logs/invoker-local_logs.log file. Logs from the controller are stored in the controller/logs/controller-local_logs.log file. All other container logs are stored in the same docker-compose.log file.

Searching these files with the activation id will allow you to find all platform logs for that action invocation.

find the runtime container for an activation

Running the platform locally means you can access runtime environments used to execute actions. This can help diagnosing and debugging application errors.

Finding the runtime container used for an invocation relies on having the activation identifier. Using this value, search the controller logs for the following log message.

[InvokerReactive] <namespace/action_id> <user> <activation_id>

This is then followed by a log message with the container identifier used for that invocation.

[DockerContainer] sending initialization to ContainerId(<CONTAINER_ID>)

Using docker exec you can then access the runtime environment to poke around!

$ docker exec -it <CONTAINER_ID> /bin/bash

install shared packages

On IBM Cloud Functions, the /whisk.system/ namespace contains shared packages for common utilities and external feed providers. These packages are not installed by default on the platform created by devtools.

These packages are available in the following repositories.

incubator-openwhisk-catalog - Shared utilities and simple feed providers.
incubator-openwhisk-package-alarms - Alarms feed provider.
incubator-openwhisk-package-cloudant - Cloudant feed provider.
incubator-openwhsk-package-kafka - Kafka feed provider.

Follow the instructions in the repositories to make these available on your local version of the platform.

Pull requests have been opened to automate installing these packages in the devtools project.

conclusion

Serverless platforms often get criticised as having a poor “developer experience” for people used to traditional application servers. Having no access to the environments running your code can make development and testing challenging. Debugging issues through console logs feels like a step backwards.

Using an open-source serverless platform means you can actually run the entire platform locally. Using a local instance during development improves deployment times, provides access to runtime environments and allows you to work offline.

Apache OpenWhisk has numerous options for deploying the platform, including virtual machines, kubernetes or native containers. Whilst the platform is not the simplest to set-up manually, the devtools project bootstraps a pre-configured version by spinning up a local instance using Docker Compose. This is the easiest approach for most developers to have a local instance of the platform.