Skip to content

New runs between running git-write.js and one of the scoring scripts will fail #157

Open
@foolip

Description

@foolip

The problem is that first we do this to populate the most recent runs:

node git-write.js --max-time=300 --max-age-days=5

The in various scoring scripts, we use the wpt.fyi API to enumerate runs:

const alignedRuns = await lib.runs.fetchAlignedRunsFromServer(
products, from, to, experimental);

And then fail if any of those aren't in the "cache":

if (!localRunIds.has(run.id)) {
// If you see this, you probably need to run git-write.js or just update
// your results-analysis-cache.git repo; see the README.md.
console.error(`Run ${run.id} missing from local git repo (${date})`);
hadErrors = true;
}

This happened in https://github.com/web-platform-tests/results-analysis/actions/runs/4240081460/jobs/7368750202

This doesn't happen often, but it will happen occasionally.

A better setup would be one of:

  • Assume the cache is sufficiently up-to-date, don't try to update it, and ignore any run not in it
  • Don't run git-write.js at all, and instead handle cache misses by creating the tree from the full wptreport.json. This would work without the results-analysis-cache repo, just very slowly.
  • Get the runs from the wpt.fyi API once, ensure those are in the cache, and then run scoring scripts (storing the runs in some intermediate JSON file)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions