Flak Cannon

AB testing framework - server

This is the Flak Cannon API server.
The web client is found here: flak-cannon-client

Install

$ npm install

Setup

Set the config.coffee environment params correctly (requires MongoDB)

$ npm start

Usage

Experiments

Creating and Editing an Experiment

Create an experiment in /experiments and add it to /experiments/index.coffee
Do not edit running experiments, instead create new ones.

The assign() method must be deterministic, parameters must be unique across all experiments

Example

picker = require 'flak-cannon-picker'

class MyExperiment

  params: ['homepage_button']
  assign: (userId) ->
    return {
      homepage_button: picker.uniformChoice(userId, ['a', 'b', 'c'])
    }


module.exports = new MyExperiment()

API

POST /experiments

Request:

{
  userId: 123
  fromUserId: 1234 // optional, forces user into the same experiment group (forever)
}

Response:

{
  login_button: 'red'
}

GET /experiments

Response:

[
  {
    id: 'login_button'
  },
  {
    id: 'another_test_parameter'
  }
]

POST /conversions

Request

{
  event: 'signup',
  uniq: 'someuniqkey', // optional, ensures uniqueness
  userId: 123
}

Response

{
  event: 'signup',
  userId: 123,
  timestamp: 'January 1, 2038',
  params: {
    login_button: 'red'
  }
}

GET /conversions

Response

[
  {
    id: 'signup'
  }
]

GET /results/?event=<event>param=<param>&from=<Date>&to=<Date>

optionally add viewCounter=dau|d7 to use a different view count aggregate

• dau counts the number of daily active users (unique by server date-time)
• d7 counts the number of signup events for 7 days ago (shifted from/to)

Response

{
  views: [
    {
      param: 'red',
      count: 123
    },
    {
      param: 'green',
      count: 113
    }
  ],
  counts: [
    [
      {
        date: 'January 1, 2038',
        value: 'red',
        count: 12
      },
      {
        date: 'January 1, 2038',
        value: 'green',
        count: 12
      },
    ],
    [
      {
        date: 'January 2, 2038',
        value: 'red',
        count: 32
      },
      {
        date: 'January 2, 2038',
        value: 'green',
        count: 22
      },
    ],
    [
      {
        date: 'January 3, 2038',
        value: 'red',
        count: 52
      },
      {
        date: 'January 3, 2038',
        value: 'green',
        count: 22
      },
    ]
  ]
}

Contributing

Run tests:

$ npm test

Run server in development mode:

$ npm run dev