Event-driven Functions

Invoke javascript functions in a Heroku app via Salesforce Platform Events.

💻👩‍🔬 This project is a exploration into a new pattern for extending the compute capabilities of Salesforce. Shared freely via MIT license. Use at your own risk.

• 📐 Design
• 🏙 Architecture
• 🌱 Example
  • Heroku function
  • Account object
  • Platform Events
  • Process Builder flows
• ✅ Requirements
• ⚡️ Usage
  • First time setup
  • Salesforce setup
  • Run locally
  • Developing more functions
• 🎛 Configuration via environment variables
  • Authentication
  • Runtime behavior
• 🔬 Testing
• 🚀 Deployment
  • Salesforce components
  • Heroku app

Design

The high-level flow is:

Platform Event → this app → Platform Event

This flow maps specific Invoke events (topics) to function calls that return values by producing Return events.

Heroku_Function_*_Invoke__e → Node.js function call → Heroku_Function_*_Return__e

These functions are composed in a Heroku app. Each function's arguments, return values, and their types must be encoded in the Invoke and Return events' fields.

Architecture

This event-driven functions app is a Node.js app, along with an sfdx project providing the Salesforce customizations.

Based on improvements to the jsforce Streaming module to support durable consumption of the Salesforce Streaming API. Those changes were merged to become jsforce 1.9.0.

Example

This repo contains an example implementation of a UUID generator for Salesforce Accounts. This could generate UUIDs for any Salesforce object by implementing a Process Builder flow for each desired object type.

Heroku function

Implemented in Node.js, an event-driven function is defined in the JavaScript module Generate_UUID and registered as a function export.

Account object

In Salesforce Setup → Object Manager, a custom field UUID__c is defined for Account.

This UUID field is blank by default, to be filled with a universally unique external indentifier generated by the Heroku app/function.

Platform Events

In Salesforce Setup → Platform Events, two events are defined.

Invoke event

Salesforce Platform Event Heroku_Function_Generate_UUID_Invoke__e

{
  "Context_Id__c": "xxxxx"
}

• Context_Id should be passed-through unchanged from Invoke to Return. It provides an identifier to associate the return value with the original invocation
• This example is a minimal Invoke event payload with no additional fields for function arguments. When defining the Platform Event in Salesforce Setup, it may contain as many fields as necessary
• This object is included in the Salesforce Permission Set for this app.

Return event

Salesforce Platform Event Heroku_Function_Generate_UUID_Return__e

{
  "Context_Id__c": "xxxxx",
  "Value__c": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx"
}

• Context_Id__c should be passed-through unchanged from Invoke to Return
• Value__c is a minimal Return event payload. In this example it contains the string UUID
• This object is included in the Salesforce Permission Set for this app.

👓 About the suffixes in Salesforce identifiers: __e is appended to Platform Event names, and __c is appended to custom object & field names.

Process Builder flows

In Salesforce Setup → Process Builder, two flows are defined.

Generate UUID for Account

The first flow triggers when creating or updating an Account that does not yet have a UUID. This publishes an Invoke event.

Set UUID for Account

The second flow triggers when a Return event is received. This updates the Account with the returned UUID.

Requirements

• Node.js 8.11 with npm 5
• redis

Usage

First time setup

git clone https://github.com/mars/event-driven-functions.git
cd event-driven-functions/
npm install
cp .env.sample .env

Salesforce setup

Next, we'll use sfdx to deploy the Salesforce customizations. If you don't yet have access to a Dev Hub org, or this is your first time using sfdx, then see Setup Salesforce DX in Trailhead.

Deploy the included force-app code to a scratch org:

sfdx force:org:create -s -f config/project-scratch-def.json -a EventDrivenFunctions
sfdx force:source:push
sfdx force:user:permset:assign -n Heroku_Function_Generate_UUID

View the scratch org description:

sfdx force:user:display

Then, update .env file with the Instance Url & Access Token values from the scratch org description:

SALESFORCE_INSTANCE_URL=xxxxx
SALESFORCE_ACCESS_TOKEN=yyyyy

⚠️ Scratch orgs and their authorizations expire, so this setup may need to be repeated whenever beginning local development work. View the current status of the orgs with sfdx force:org:list.

Run locally

Open the scratch org's Accounts:

sfdx force:org:open --path one/one.app#/sObject/Account/list

Run this node command in a shell terminal:

READ_MODE=changes \
PLUGIN_NAMES=invoke-functions \
OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e \
node lib/exec

🔁 This command runs continuously, listening for the Platform Event.

▶️ Watch this command's output as your create Accounts in the scratch org.

Developing more functions

For a given function, three identifiers are used.

• Function Name
  • example Generate_UUID
  • used for the JavaScript module file & its functions export
• Invoke Event Name
  • example Heroku_Function_Generate_UUID_Invoke__e
  • used for the Platform Event that runs the function
• Return Event Name
  • example Heroku_Function_Generate_UUID_Return__e
  • used for the Platform Event that receives the function's result

Note: the Function Name must be embedded exactly in both Event Names.

To implement a new function:

1. create the new Platform Events using sfdx workflow
   • develop in a scratch org and pull changes into this repo
   • define each Invoke & Return Event and its schema (fields & their types)
   • define a new Permission Set for access or add to an existing Set
2. create the function as a default export in lib/functions/
   • use Function Name for the module file & its functions export
   • the function receives the Invoke event's payload and must honor the Return event's schema
   • see: example Generate_UUID.js & the export
3. include the new Invoke Event Name in OBSERVE_SALESFORCE_TOPIC_NAMES env var
   • example: OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e,/event/Heroku_Function_Generate_Haiku_Invoke__e

Configuration

Configure Authentication

Performed based on environment variables. Either of the following authentication methods may be used:

• Username + password
  • SALESFORCE_USERNAME
  • SALESFORCE_PASSWORD (password+securitytoken)
  • SALESFORCE_LOGIN_URL (optional; defaults to login.salesforce.com)
• Existing OAuth token

  • SALESFORCE_INSTANCE_URL

  • SALESFORCE_ACCESS_TOKEN

  • Retrieve from an sfdx scratch org with:

    sfdx force:org:create -s -f config/project-scratch-def.json -a EventDrivenFunctions
    sfdx force:org:display

• OAuth client
  • SALESFORCE_URL
    • Must include oAuth client ID, secret, & refresh token
    • Example: force://{client-id}:{secret}:{refresh-token}@{instance-name}.salesforce.com

Configure Runtime Behavior

• VERBOSE
  • enable detailed runtime logging to stderr
  • example: VERBOSE=true
  • default value: unset, no log output
• PLUGIN_NAMES
  • configure the consumers/observers of the Salesforce data streams
  • example: PLUGIN_NAMES=invoke-functions
  • default value: console-output
• OBSERVE_SALESFORCE_TOPIC_NAMES
  • the path part of a Streaming API URL
  • a comma-delimited list
  • example: OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e
  • default value: no Salesforce observer
• REDIS_URL
  • connection config to Redis datastore
  • example: REDIS_URL=redis://localhost:6379
  • default: unset, no Redis
• REPLAY_ID
  • force a specific replayId for Salesforce Streaming API
  • ensure to unset this after usage to prevent the stream from sticking
  • example: REPLAY_ID=5678 (or -2 for all possible events)
  • default: unset, receive all new events

Testing

Implemented with AVA, concurrent test runner.

npm test runs only unit tests. It skips integration tests, because Salesforce and AWS config is not automated.

Unit Tests

• npm run test:unit
• Defined in lib/ alongside source files
• Salesforce API calls are mocked by way of Episode 7

Deployment

Deploy Salesforce components

Example, ready-to-install package from this repo

sfdx force:auth:web:login -a AnotherOrg
sfdx force:package:install --id 04tf4000001ft4hAAA -u AnotherOrg

Custom, package it yourself

2nd-generation unnamespaced, unlocked package (2GP)

Based on Salesforce DX 2GP docs:

sfdx force:package2:create --name Event_Driven_Functions_Generate_UUID_2GP_sans_NS --description "Integration with event-driven-functions Heroku app" --containeroptions Unlocked --nonamespace

In sfdx-project.json, update packageDirectories.0.id with the output Package2 Id:

{
  "packageDirectories": [
    {
      "path": "force-app",
      "default": true,
      "id": "0Hof4000000blNuCAI",
      "versionName": "Initial 2GP",
      "versionNumber": "1.0.0"
    }
  ],
  "namespace": "",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "42.0"
}

sfdx force:package2:version:create --directory force-app --wait 10
sfdx force:package:install --id <Subscriber Package2 Version Id> -u OrgAliasOrUserId --wait 10 --publishwait 10
sfdx force:org:open -u EventDrivenFunctions

1st-generation unmanaged (unnamespaced) package

Follow Build and Release Your App with Managed Packages to prepare a packaging org.

Diverging from those directions, we'll prepare an unmanaged package without a namespace. We have to skip namespacing for now, because of problems with Process Builder + Platform Events embedding namespaces in the metadata (evidence 1, 2). Link its namespace with your Hub org, and set the established "namespace" in sfdx-project.json, and then provision & push to a fresh scratch org.

Now, pull the Salesforce customizations back out of the scratch org in the Metadata API format:

sfdx force:source:convert --outputdir mdapi_output_dir --packagename Event_Driven_Functions_Generate_UUID

Login to the packaging org and create the Beta package:

sfdx force:org:list
sfdx force:auth:web:login -a PkgFunctions

sfdx force:mdapi:deploy --deploydir mdapi_output_dir --targetusername PkgFunctions

# Find the package ID in the URL of the packaging org:
#   Setup → Package Manager → View/Edit the Package
sfdx force:package1:version:create --packageid XXXXX --name r00000 -u PkgFunctions

sfdx force:package1:version:list -u PkgFunctions

Install the beta package into another org by its id METADATAPACKAGEVERSIONID:

sfdx force:auth:web:login -a AnotherOrg
sfdx force:package:install --id YYYYY -u AnotherOrg

Deploy Heroku app

heroku create

heroku config:set \
  [email protected] \
  SALESFORCE_PASSWORD=nnnnnttttt \
  VERBOSE=true \
  PLUGIN_NAMES=invoke-functions \
  OBSERVE_SALESFORCE_TOPIC_NAMES=/event/Heroku_Function_Generate_UUID_Invoke__e \
  READ_MODE=changes

heroku addons:create heroku-redis:premium-0

git push heroku master

heroku ps:scale web=0:Standard-1x worker=1:Standard-1x