Falafel

A Node.js framework for making it crazy easy to build connectors. Built on top of threadneedle allowing for a declarative operation based approach.

Falafel uses JavaScript-based schemas as a superset of connectors.json, but unlike connectors.json, the schema also has a direct impact on the running of operations. For example, the use of required makes a field required in connectors.json as well as on the operational level.

Get started now View it on GitHub

Table of contents

1. Project structuring
   1. Connector file
   2. Operations (previously Messages)
   3. Schema
   4. Model
      1. after_headers
   5. Sample response
2. Global models
3. Private methods
4. Dynamic output schemas
5. Testing the connector
   1. As a server/exposed function
   2. Writing tests

Project structuring

Falafel requires you to follow a strict folder structure for organising connectors:

connectors/
  connectorname/
    my-message/
      model.js
      schema.js
      response.sample.json
    my-second-message/
      model.js
      schema.js
      response.sample.json
    connector.js
    global_model.js (optional)
    global_schema.js (optional)

Connector file

The connector.js file contains high level config about the connector, mostly related to how the connector appears in the builder UI.

module.exports = {

 // Title as it will appear in the builder UI
 title: 'MailChimp',

 // Description as it will appear in the builder UI
 description: 'Interact with the MailChimp API.',

 // Version of the schema
 version: '1.0',

 // Tags
 tags: ['service'],

 // Icon
 icon: {
   type: 'url',
   value: 'http://images.tray.io.s3.amazonaws.com/static/icons/placeholder.png',
 },

};

Operations (previously Messages)

On a high level, the following rules apply for each message.

• The schema.js handles the input schema
• The model.js handles the running of the operation
• The response.sample.json provides a sample output - for the output schema

Schema

The schema.js file is a higher level JavaScript version of the connectors.json file. It takes a declarative approach, allowing for inline advanced and required variables, auto-generating message and property title attributes, and allowing for JavaScript utility functions:

module.exports = {

  input: {

    access_token: {
      type: 'string',
      advanced: true,
      required: true,
      defaultJsonPath: '$.auth.access_token'  
    },

    id: {
      type: 'string',
      description: 'The MailChimp list ID.',
      required: true
    }

  })

};

This schema will be used to generate the input_schema for each message. Also, the required variable applies a validation before the operation executes at runtime.

type is a top level schema.js property for which the allowed values are public, ddl, and private. If type is not provided, the public is assumed by default, unless the operation name ends with _ddl. Only public and ddl operation types are added to the connectors.json.

Model

Any options in the model.js file will be automatically be passed to a threadneedle method for the operation. For example:

module.exports = {

  url: 'https://.api.mailchimp.com/2.0/lists/list?apikey=',
  method: 'get',
  expects: 200

};

Variables passed in the input schema will be passed into a Mustache template system.

after_headers

For function models (only), the afterHeaders function can be specified in a after_headers.js file, exporting a function accepting the arguments error, body, and params.

module.exports = function (error, params, body) {
	return {
	//Must return an object
	};
};

Sample response

Output schemas are important in tray - they allow connectors to reference the
data coming from a previous connector. However, you don’t need fine grained control, handling variables like required and advanced.

Falafel means you don’t have to explicitly declare an output schema for each message. Just add a response.sample.json file for each and a JSON schema generator will automatically generate an output schema when building the connectors.json.

Global models

Threadneedle has a “global models” approach which allows for shared logic across multiple messages. If you declare the connectors/myconnector/global_model.js (previously global.js) file, the options in it will be globalized for the connector across all methods:

module.exports = {

  before: function (params) {
    params.dc = 'us5';
    // you can also return a promise that should resolve after modifying `params`
  }

};

Tip: If you’d like to disable global logic for an operation, just set globals: false in the model.js config file.

See the threadneedle docs for more information on globals.

Private methods

Sometimes you’ll want to create an internal method that should not be exposed to the UI. Typically the main use for this will be a generic method called in before, providing key data to enable the main method to run.

This is simple - just don’t add the schema.js and response.sample.json files in the message folder.

Note: the operation will be still be created, but it won’t be added to the connectors.json config (so won’t appear in the UI).

Dynamic output schemas

If an operation is to support dynamic output schemas, an output.js file simply needs to be included in the operation’s folder. The file should always export a promise function to be run, like so:

module.exports = function (params) {
	return when.promise(function (resolve, reject) {
		//Return an object or JSON schema
	});
};

Falafel accepts two possibilities for the returned data; either an object or a JSON schema. A JSON schema is identified by having a "$schema" key/property on the top level; thus, if detected, the data will be passed on directly. Otherwise, Falafel will transform the object to JSON schema and then pass it.

Note: Depending on whether a output.js is included or not, Falafel will automatically set the dynamic_output key in connectors.json for each operation; thus dynamic_output attribute does not need to be added in schema.js. Additionally, the dynamic output sub-operation can be referenced as message_dynamic_output.

Testing the connector

As a server/exposed function

Running the connector with NODE_ENV set to development as an environment variable will spin up a testing HTTP server, to which request to connector operations can be made via a tool like Postman.

HTTP requests are sent to http://localhost:8989/send/123-def with a body format like:

{
  "id": "123-def",
  "header": {
    "message": "[OPERATION NAME]"
  },
  "body": {
    "...[INPUT PARAMETERS]"
  }
}

The id 123-def is used when testing.

Writing tests

Generally, sub-operations like request.js and output.js are not exposed in the falafel['connector name'] object. However, if access is needed for when writing test scripts, setting NODE_ENV to test as an environment variable will add the sub-operations to the falafel['connector name'] object.