Implement multi-mine aggregated queries (say, for QTL)

[sammyjava]

A reminder for me to work on this.

[sammyjava]

https://github.com/legumeinfo/graphql-server/tree/multiple-mines is an example solution, perhaps not properly done.

[alancleary]

Sorry for the delay on this; I finally have some feedback:

I think the aggregate resolvers themselves seem fine but they shouldn't be located in the intermine resolver factories. Each factory generates resolvers for a specific source, i.e. the sourceName parameter, so generating resolvers that aggregate data from all sources breaks the design pattern. The resolvers barrel where you use the factories to instantiate resolvers for multiple mines would be the place for aggregate resolvers.

Are the "multi" resolvers meant to replace their non-multi counterparts or are they intended to be in the API? I think the latter is less ideal because it's revealing information about the implementation in the interface. The more ideal scenario would be to provide a single resolver that is multi by default but allows users to unwittingly winnow down the set of sources they're loading data from via parmaeters.

[sammyjava]

Good feedback. I shall ponder and try some stuff out!

[sammyjava]

Are the "multi" resolvers meant to replace their non-multi counterparts or are they intended to be in the API? I think the latter is less ideal because it's revealing information about the implementation in the interface. The more ideal scenario would be to provide a single resolver that is multi by default but allows users to unwittingly winnow down the set of sources they're loading data from via parmaeters.

Well, the idea that I had was that for genomic queries we'd stick with a single mine, LegumeMine, which I had configured as the "default" mine. But for genetic queries, we have to go to a different mine - LegumeMine doesn't have genetic data - and each genus mine has its own genetic data. So in that case we have to either pick a mine (bad) or query across genetic mines (better).

That's the goal, I don't care a lot HOW we get there. But, put simply, you cannot query a QTL against LegumeMine. There are no QTLs in LegumeMine. But ALL the LIS genes and proteins and gene family assignments are in LegumeMine. So we have to be able to query other mines if we're going to query any genetic data.

The other option would be to switch to a single, huge mine. There has always been push-back against that.

[sammyjava]

The more ideal scenario would be to provide a single resolver that is multi by default but allows users to unwittingly winnow down the set of sources they're loading data from via parmaeters.

I may need help with that. I've never implemented "unwitting" functionality in a UI. :)

[alancleary]

It sounds like what we want to do then is make LegumeMine's resolvers the default API and overwrite certain endpoints to use a multi implementation. This means the actual GraphQL API won't change.

Here's a code example that I haven't tested:

// src/resolvers/index.html

const { mergeResolvers } = require('@graphql-tools/merge');
const intermineResolverFactory = require('./intermine');

// genus mines with genetic data as well
const geneticMines = [
  'lisArachisMineAPI',
  'lisGlycineMineAPI',
  'lisPhaseolusMineAPI',
  'lisVignaMineAPI',
];

// genus mines with only genomic data

const genomicMines = [
  ...geneticMines,
  'lisAeschynomeneMineAPI',
  'lisCajanusMineAPI',
  'lisCicerMineAPI',
  'lisLensMineAPI',
  'lisLupinusMineAPI',
  'lisMedicagoMineAPI',
];

const resolvers = [
  // across-the-board genomic data
  intermineResolverFactory('lisLegumeMineAPI'),
  // add more resolvers here
  // NOTE: you can't use mergeResolvers with multiple intermine resolvers; you'll
  // have to implement an aggregate resolver so they don't overwrite each other
  {
    Query: {
      gwases: async (_source, { description, start, size }, { dataSources }) => {
        const args = {description, start, size};
        return genomicMines.map((sourceName) => {
            return dataSources[sourceName].searchGWASes(args);
        });
      },
    },
  }
];

module.exports = mergeResolvers(resolvers);

The Query object and its gwases property (intentionally) collide with resolvers generated by intermineResolverFactory('lisLegumeMineAPI'). mergeResolvers merges resolvers via a deep copy of the JavaScript objects. Whenever a collision occurs, the later instance will be used. In this case, the multi resolver.

[sammyjava]

Cool I'll give that a try in a new branch.