Last month, a colleague was explaining he was not looking forward to an afternoon of long-distance conference calls. Having recently started using Slack for collaboration with their remote team, they lamented…
I wish I could do my conference calls using Slack!
…which got us thinking.
Recent experiments with IBM Watson Speech To Text and Twilio on IBM Bluemix had shown how easy it was to create telephony applications. Slack publishes multiple APIs to help developers build custom “bots” that respond to channel content.
Could we create a new Slackbot that let users make phone calls using channel messages?
One month later, Phonebot was born!
Slackbot that lets users make phone calls within a Slack channel.
Users can dial a phone number, with the phone call audio converted to text and sent to the channel.
Channel message replies are converted to speech and sent over the phone call.
Read on to find out how we put IBM Watson, Twilio and IBM Bluemix to develop our custom Slackbot…
Phonebot will listen to messages starting with @phonebot and which contain user commands e.g. dial, hangup. It will create new channel messages with the translated speech results along with status messages. Users can issue the following commands to control Phonebot.
1 2 3 4 5 6
Listening for custom commands
Creating a new Outgoing Webhook, messages from the registered channels which begin with the “@phonebot” prefix will be posted to HTTP URL for the IBM Bluemix application handling the incoming messages.
We can create Outgoing Webhooks for every channel we want to register Phonebot in.
For each registered channel, we need to allow Phonebot to post new messages.
Sending new channel messages
Incoming Webhooks provide an obfuscated HTTP URL that allows unauthenticated HTTP requests to create new channel messages. Creating a new Incoming obfuscated for each channel we are listening to will allow Phonebot to post responses.
Each Incoming Webhook URL will be passed to Phonebot application using configuration via environment variables.
Making Phone Calls
Twilio provides “telephony-as-a-service”, allowing applications to make telephone calls using a REST API.
Twilio has been made available on the IBM Bluemix platform. Binding this service to your application will provide the authentication credentials to use with the Twilio client library.
When users issue the “call” command with a phone number, the channel bot listening to user commands emits a custom event.
1 2 3 4 5 6 7 8 9 10
Within the “phone” object, the “call” method triggers the following code.
1 2 3 4 5 6 7 8 9 10 11 12
The URL parameter provides a HTTP URL which Twilio will use to POST updated call status information. HTTP responses from this location will tell Twilio how to handle the ongoing call, e.g. play an audio message, press the following digits, record phone line audio.
If the phone call connects successfully, we need the phone line audio stream to translate the speech into text. Unfortunately, Twilio does not support directly accessing the real-time audio stream. However, can record a batch of audio, i.e five seconds, and download the resulting file.
Therefore, we will tell Twilio to record a short section of audio and post the results back to our application. When this message is received, our response will contain the request to record another five seconds. This approach will provide a semi-realtime stream of phone call audio for processing.
Here is the code snippet to construct the TwiML response to record the audio snippet. Any channel messages that are queued for sending as speech will be added to the outgoing response.
1 2 3 4 5 6 7 8 9 10
When we have the audio files containing the phone call audio, we can schedule these for translation with the IBM Watson Speech To Text service.
Translating Speech To Text
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
This module relies on the SOX library being installed in the native runtime. We used a custom buildpack to support this.
Managing Translation Tasks
When a new Twilio message with audio recording details is received, we schedule a translation request. As this background task returns, the results are posted into the corresponding Slack channel.
If a translation request takes longer than expected, additional requests may be scheduled before the first has finished. We still want to maintain the order when posting new channel messages, even if later requests finishing translating first.
Each time the phone object for a channel emits a ‘recording’ event, we start the translation request and post the worker to the channel queue.
1 2 3 4 5 6 7 8
When a task reaches the front of the queue, the worker function is called to process the result.
If translation task has finished, we signal to the queue this task has completed. Otherwise, we wait for completion events being emitted.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Now we’ve finished the code, we can configure the application to deploy on the IBM Bluemix cloud platform.
Phonebot must be passed the configured incoming webhooks URLs, allowing it to send channel messages. Following the standard Platform-as-a-Service convention for passing configuration, we store the channel webhooks as environment variables.
Using the CF CLI, we run the following command to set up the local environment parameters.
Application manifests configure deployment parameters for Cloud Foundry applications. Phonebot will need to be bound to Twilio, IBM watson and custom services, along with configuring the runtime environment.
--- applications: - name: phonebot memory: 256M command: node app.js buildpack: https://github.com/jthomas/nodejs-buildpack.git services: - twilio - speech_to_text - slack_webhooks declared-services: twilio: label: Twilio plan: 'user-provided' twilio: label: slack_webhooks plan: 'user-provided' speech_to_text: label: speech_to_text plan: free
…with this manifest, we can just use the cf push command to deploy our application!
Phonebot will post the following message to each channel successfully registered on startup.
Users can issue @phonebot COMMAND messages to control phone calls directly from the slack channel.
For further information about the project, follow the project on Github. Upcoming features are listed in the issues page. Please feel free to ask for new features, report bugs and leave feedback on Github.