Skip to content
This repository was archived by the owner on Jul 17, 2020. It is now read-only.

Solution Overview

Jump to bottom
johnadamsDFID edited this page Nov 29, 2013 · 9 revisions

High Level Description

The DevTracker platform aims to provide transparent access to UK International Development Spend from UK government and partners in a visual manner.

Architecture Overview

The currently deployed solution looks roughly like this,

As Is

There is an alternative model currently in alpha, using BaseX and a standard API.

Data Flow

The admin site is the main control point for the system. This set of steps covers what needs done on a new/fresh machine.

  • Admin user logs into the admin functionality
  • User selects Organisation sources by first loading them (Refresh Sources) and then selecting them and hitting save
  • User does the same for activity sources
  • User updates the CMS content (country/region names etc.) - this data will already be seeded from the migration step of the install but there may be extra changes if the user desires.
  • Select "Load Data". This downloads, validates and loads selected Org/Activity sources into Neo4J/Mongo, indexes the data for search purposes and exposes the new data via the APi
  • After the data is fully loaded the user can select to "Deploy the Site" - this process takes data aggregated into Mongo and builds the static site.

As Is Flow

Some basic data flow visualisations:

As Is

Architecture layers

The current architecture contains the following elements:

Content Management System (CMS) src/platform/modules/admin

The CMS admin is a user friendly approach to defining what IATI XML files should be loaded as well as providing an area to add static descriptions to countries and regions and other content.

Loader and Aggregator src/platform/modules/loader

The loader pulls in the IATI XML data and loads it into the data stores of the API and Search. The application uses two data stores:

  • Neo4J - this graph database stores all the IATI XML data in nodes that match the IATI XML structure. This is then interrogated by the Aggregator.
  • Mongo - the Aggregator moves data from the Neo4J graph database into a set of Mongo tables that match the key data structures we need for the site, in particular:
    • db.projects - DFID projects
    • db.other-org-projects - projects for other UK government departments
    • db.funded-projects - partner projects for traceability

Searcher src/platform/modules/search

Provides faceted, fuzzy searching of loaded IATI data using Elastic Search. This is only available to the Platform and not through the API. The rationale for doing this is that it is very much geared towards the aid-platform format rather than arbitrary API calls.

API src/platform/app/controllers

IATI Data API built against the working draft of the IATI API Standards.

Site src/platform/site

The aid platform uses a set of Ruby templates to generate a static site that can be hosted on any provider. Data is taken from the Mongo database.

Clone this wiki locally