Creating Swift Binaries for OpenWhisk

In the previous blog post, we explained how to write Serverless Swift functions using OpenWhisk actions.

Swift sources files are compiled into a binary by the platform before processing requests.

This compilation process adds a delay on the invocation time for “cold” runtimes. If the action has not been invoked for a while, the system is under heavy load or multiple invocations are received in parallel, a new runtime will need to be initialised.

Pre-compiled binaries can be deployed to remove this delay. Binaries must be compiled for the correct platform architecture and support execution through the OpenWhisk runtime.

There is now a Swift package to make the process of building pre-compiled binaries much easier.

Let’s have a look at how this works…

Swift Packages

Swift introduced a package manager in Swift 3.0. The package manager integrates with the build system to “automate the process of downloading, compiling, and linking dependencies”.

Swift uses a manifest file (Packages.swift) to define package properties including dependencies.

Example Swift Package

Here’s an example manifest file from a sample package with external dependencies.

import PackageDescription

let package = Package(
    name: "DeckOfPlayingCards",
    targets: [],
    dependencies: [
        .Package(url: "https://github.com/apple/example-package-fisheryates.git",
                 majorVersion: 1),
        .Package(url: "https://github.com/apple/example-package-playingcard.git",
                 majorVersion: 1),
    ]
)

Packages are referenced through a URL which resolves to a Git repository. Semantic versioning conventions are used to control the package version installed.

External packages are downloaded, compiled and linked in the project during the build process.

$ swift build
Fetching https://github.com/apple/example-package-deckofplayingcards.git
Fetching https://github.com/apple/example-package-fisheryates.git
Fetching https://github.com/apple/example-package-playingcard.git
Cloning https://github.com/apple/example-package-fisheryates.git
Resolving https://github.com/apple/example-package-fisheryates.git at 2.0.3
Cloning https://github.com/apple/example-package-playingcard.git
Resolving https://github.com/apple/example-package-playingcard.git at 3.0.2
Cloning https://github.com/apple/example-package-deckofplayingcards.git
Resolving https://github.com/apple/example-package-deckofplayingcards.git at 3.0.3
Compile Swift Module 'PlayingCard' (3 sources)
Compile Swift Module 'FisherYates' (2 sources)
Compile Swift Module 'DeckOfPlayingCards' (1 sources)
Compile Swift Module 'Dealer' (1 sources)
Linking ./.build/debug/Dealer
$

OpenWhiskAction Package

OpenWhiskAction is a Swift package for registering Swift functions as OpenWhisk actions.

It bundles the Swift source files used to implement the runtime handler for OpenWhisk as a library. Using this package means developers do not have to manually import those source files into their projects.

usage

This package exposes a public function (OpenWhiskAction ) that should be called with a function reference (([String: Any]) -> [String: Any])) as a named parameter (main). The callback will be executed with the invocation parameters. Returned values will be serialised as the invocation response.

import OpenWhiskAction

func hello(args: [String:Any]) -> [String:Any] {
    if let name = args["name"] as? String {
      return [ "greeting" : "Hello \(name)!" ]
    } else {
      return [ "greeting" : "Hello stranger!" ]
    }
}

OpenWhiskAction(main: hello)

example

Let’s show an example of using the package to build a pre-compiled Swift action for OpenWhisk.

create new package

Create a new directory and use the swift package init command to generate the boilerplate package.

$ mkdir Action
$ cd Action/
$ swift package init
Creating library package: Action
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/Action.swift
Creating Tests/
Creating Tests/LinuxMain.swift
Creating Tests/ActionTests/
Creating Tests/ActionTests/ActionTests.swift

add package dependency

Add the OpenWhiskAction package as a dependency to the manifest file (Package.swift).

import PackageDescription

let package = Package(
    name: "Action",
    dependencies: [
        .Package(url: "https://github.com/jthomas/OpenWhiskAction.git", majorVersion: 0)
    ]
)

write serverless function

Create a new main.swift file under the Sources directory containing the following source code.

import OpenWhiskAction

func hello(args: [String:Any]) -> [String:Any] {
    if let name = args["name"] as? String {
      return [ "greeting" : "Hello \(name)!" ]
    } else {
      return [ "greeting" : "Hello stranger!" ]
    }
}

OpenWhiskAction(main: hello)

Swift’s build process will produce an executable if the package contains a main.swift file. That file will be compiled as the package binary.

compiling with docker

OpenWhisk Swift actions use a custom Docker image as the runtime environment. Compiling application binaries from this image will ensure it is compatible with the platform runtime.

This command will run the swift build command within a container from this image. The host filesystem is mounted into the container at /swift-package. Binaries and other build artifacts will be available in ./.build/release/ after the command has executed.

docker run --rm -it -v $(pwd):/swift-package openwhisk/action-swift-v3.1.1 bash -e -c "cd /swift-package && swift build -v -c release"

deploying to openwhisk

OpenWhisk actions can be created from a zip file containing action artifacts. The zip file will be expanded prior to execution. In the Swift environment, the Swift binary executed by the platform should be at ./.build/release/Action.

If an action is deployed from a zip file which contains this file, the runtime will execute this binary rather than compiling a new binary from source code within the zip file.

$ zip action.zip .build/release/Action
  adding: .build/release/Action (deflated 67%)
$ wsk action create swift-action --kind swift action.zip
ok: created action swift-action
$ wsk action invoke --blocking --result -p name "Bernie Sanders" swift-action
{
    "greeting": "Hello Bernie Sanders!"
}

Using With The Serverless Framework

As shown in the previous blog post, The Serverless Framework supports the Swift runtime. Actions can either be created from Swift source files or pre-compiled binaries.

This example project demonstrates how to integrate pre-compiled binaries into a serverless framework application.

example project

The project contains two Swift source files under the Sources directory. Using the main.swift file name means these files will be compiled into separate binaries under the .build/release directory.

$ tree .
.
├── Package.swift
├── README.md
├── Sources
│   ├── hello
│   │   └── main.swift
│   └── welcome
│       └── main.swift
├── package.json
└── serverless.yml

3 directories, 6 files

The package manifest (Package.swift) contains the OpenWhiskAction dependency.

serverless.yml

service: swift-packages

provider:
  name: openwhisk
  runtime: swift

functions:
  hello:
    handler: .build/release/hello
  welcome:
    handler: .build/release/welcome

custom:
  scripts:
    hooks:
      'package:initialize': npm run-script compile
plugins:
  - serverless-openwhisk
  - serverless-plugin-scripts

This configuration file describes two actions (hello and welcome) using the swift runtime. The handler property for those actions refers to a binary, produced by the build process, rather than source file.

compile during deployment

Before using serverless deploy command to create our application, we need to compile binaries for the OpenWhisk runtime.

Manually running the Swift build command before each deployment is cumbersome and error-prone.

Let’s automate this process…

Using NPM’s scripts feature, the project exports a new command npm run-script compile which triggers the build process using the OpenWhisk Docker runtime for Swift.

{
  "name": "openwhisk-swift-package-with-precompiled-binaries",
  "version": "1.0.0",
  "description": "Swift packages and pre-compiled binaries on OpenWhisk.",
  "main": "handler.js",
  "scripts": {
    "postinstall": "npm link serverless-openwhisk",
    "compile": "docker run --rm -it -v $(pwd):/swift-package openwhisk/action-swift-v3.1.1 bash -e -c 'cd /swift-package && swift build -v -c release'"
  },
  "keywords": [
    "serverless",
    "openwhisk"
  ],
  "dependencies": {
    "serverless-plugin-scripts": "^1.0.2"
  }
}

The serverless-plugin-scripts plugin provides a mechanism for running shell commands when framework commands are executed. Using this plugin we can use the package:initialize event to execute the npm run-script compile command.

custom:
  scripts:
    hooks:
      'package:initialize': npm run-script compile

The package:initialize event is fired when the serverless deploy command executes.

Swift binaries will be compiled prior to deployment without any manual steps from the developer.

$ serverless deploy
> openwhisk-swift-package-with-precompiled-binaries@1.0.0 compile /Users/james/code/bluemix/serverless-examples/openwhisk-swift-precompiled-binaries
> docker run --rm -it -v $(pwd):/swift-package openwhisk/action-swift-v3.1.1 bash -e -c 'cd /swift-package && swift build -v -c release'
...
Serverless: Packaging service...
Serverless: Compiling Functions...
Serverless: Compiling API Gateway definitions...
Serverless: Compiling Rules...
Serverless: Compiling Triggers & Feeds...
Serverless: Deploying Functions...
Serverless: Deployment successful!

Service Information
platform:	openwhisk.ng.bluemix.net
namespace:	_
service:	swift-packages

actions:
swift-packages-dev-hello    swift-packages-dev-welcome
...
$ serverless invoke -f hello
{
    "greeting": "Hello stranger!"
}
$ serverless invoke -f welcome
{
    "greeting": "Welcome stranger!"
}

Conclusion

OpenWhisk supports creating Swift actions from source files and pre-compiled binaries. Using binaries reduces the startup time for “cold” environments. This is important for latency sensitive applications like API endpoints.

Swift binaries for OpenWhisk must be compiled for the correct architecture and support execution through the platform runtime. Previous instruction for producing these binaries involved numerous manual and error-prone steps.

This process has now been improved through a new Swift package which wraps the runtime handler source files. Adding this dependency into the package manifest file means the downloading, compiling and linking of these source files will be handled by the Swift package manager.

Recent updates to the OpenWhisk provider plugin for The Serverless Framework also added support for pre-compiled Swift binaries. Combined with other plugins, the framework can now completely automate the process of building binaries for the Swift runtime.

Building binaries for Swift OpenWhisk actions has never been easier! 😎