Create custom analysis pipelines

You can start any analysis activity on a resource with:

node scripts/manage.js --task=tasks.resource.discoverOne --id=17

This creates or merges appears_in relationship between a node resource <id 17> to the entities received by the chosen services.

See below a quick overview of such a process:

1. models/resource.js

The method discover() at line #L652 concatenates the text from the fields specified in settings.disambiguation.services and sends it to the helpers[service]line #873 a function that handles the language. Functions are available in helpers.jsmodule. In settings.js, the property settings.disambiguation.services specifies the service name (given as dict key) and the list of supported languages (array of 2chars language).

2. helpers.js helpers[service] should point to an existing function that maps the results of a "service" (remote or local) to a list of entities modelled in this more convenient format:

entity = {
  name: "name of the entity",
  links_wiki: "wiki_resource_if_provided",
  context:[
    left: 123, // leftmost character where the entity is found
    right: 140, // rightmost ...
    matched_text: "text of the entity", //real portion of text that matches the `name`"
  ],
  type: ["topic"], // or whatever type
}

3. services.js Put in this module the remote services to handle requests errors/results and fill the related helpers function.