CouchDB Filters with OpenWhisk Triggers

Imagine you have an OpenWhisk action to send emails to users to verify their email addresses. User profiles, containing email addresses and verification statuses, are maintained in a CouchDB database.

{
    ...
    "email": {
        "address": "user@host.com",
        "status": "unverified"
    }
}

Setting up a CouchDB trigger feed allows the email action to be invoked when the user profile changes. When user profiles have unverified email addresses, the action can send verification emails.

Whilst this works fine - it will result in a lot of unnecessary invocations. All modifications to user profiles, not just the email field, will result in the action being invoked. This will incur a cost despite the action having nothing to do.

How can we restrict document change events to just those we care about?

CouchDB filter functions to the rescue 🦸‍♂️🦸‍.

CouchDB Filter Functions

Filter functions are Javascript functions executed against (potential) change feed events. The function is invoked with each document update. The return value is evaluated as a boolean variable. If true, the document is published on the changes feed. Otherwise, the event is filtered from the changes feed.

example

Filter functions are created through design documents. Function source strings are stored as properties under the filters document attribute. Key names are used as filter identifiers.

Filter functions should have the following interface.

function(doc, req){   
    // document passes test
    if (doc.property == 'value'){
        return true;
    }
    
    // ... else ignore document upate
    return false;
}

doc is the modified document object and req contains (optional) request parameters.

Let’s now explain how to create a filter function to restrict profile update events to just those with unverified email addresses…

Filtering Profile Updates

user profile documents

In this example, email addresses are stored in user profile documents under the email property. address contains the user’s email address and status records the verification status (unverified or verified).

When a new user is added, or an existing user changes their email address, the status attribute is set to unverified. This indicates a verification message needs to be sent to the email address.

{
    ...
    "email": {
        "address": "user@host.com",
        "status": "unverified"
    }
}

unverified email filter

Here is the CouchDB filter function that will ignore document updates with verified email addresses.

function(doc){   
    if (doc.email.status == 'unverified'){
        return true;
    }
    
    return false
}

design document with filters

Save the following JSON document in CouchDB. This creates a new design document (profile) containing a filter function (unverified-emails).

{
  "_id": "_design/profile",  
  "filters": {
    "unverified-emails": "function (doc) {\n  if (doc.email.status == 'unverified') {\n    return true\n  }\n  return false\n}"
  },
  "language": "javascript"
}

trigger feed with filter

Once the design document is created, the filter name can be used as a trigger feed parameter.

wsk trigger create verify_emails --feed /_/myCloudant/changes \
--param dbname user_profiles \
--param filter "profile/unverified-emails"

The trigger only fires when a profile change contains an unverified email address. No more unnecessary invocations, which saves us money! 😎

caveats

“Why are users getting multiple verification emails?" 😡

If a user changes their profile information, whilst leaving their email address the same but before clicking the verification email, an additional email will be sent.

This is because the status field is still in the unverified state when the next document update occurs. Filter functions are stateless and can’t decide if this email address has already been seen.

Instead of leaving the status field as unverified, the email action should change the state to another value, e.g. pending, to indicate the verification email has been sent.

Any further document updates, whilst waiting for the verification response, won’t pass the filter and users won’t receive multiple emails. 👍

Conclusion

CouchDB filters are an easy way to subscribe to a subset of events from the changes feed. Combining CouchDB trigger feeds with filters allows actions to ignore irrelevant document updates. Multiple trigger feeds can be set up from a single database using filter functions.

As well as saving unnecessary invocations (and therefore money), this can simplify data models. A single database can be used to store all documents, rather than having to split different types into multiple databases, whilst still supporting changes feeds per document type.

This is an awesome feature of CouchDB!