vg chunk failed: error[vg chunk]: input path not found in xg index

[SimonaSecomandi]

Hi all,

I am trying to visualize a small pangenome chunk (368 KB). I have a .vg file and .xg file generated as follows:

vg chunk -t 8 -c 1 -x 2_bTaeGut_pangenome/bTaeGut_pangenome.xg -p bTaeGut7_mat#0#chr22:0-100000 -O vg > 3_stats_and_viz/bTaeGut_pangenome.chunk.100Kb.vg

vg convert -t 8 -x 3_stats_and_viz/bTaeGut_pangenome.chunk.100Kb.vg > 3_stats_and_viz/bTaeGut_pangenome.chunk.100Kb.xg

I followed the exact instructions explained here #446 to upload a custom file in the online demo, but I'm still getting this error:

vg chunk failed: error[vg chunk]: input path bTaeGut7_mat#0#chr22 not found in xg index

Indeed, the paths don't load at all. I tried doing the same on localhost and it worked, the paths are there. However, I would really need the online demo for a course I am preparing.

Many thanks!

Best,

Simona

[adamnovak]

I tried doing the same on localhost and it worked, the paths are there.

That sounds like we need to update the demo server and refresh the tube map and vg binary versions.

Does it work any better with the current setup if you send the .vg file to the server instead of converting it to .xg first? You should no longer need to do that.

[adamnovak]

OK @SimonaSecomandi, I've updated the tube map with the current code version and the most recent vg release, which I think has everything important about subpath handling, and which is likely to be the vg you are using. Try it now.

I'd also caution you to have a backup plan for the course if, on the day, the tube map demo server happens to not be working. Since it's "just" a demo, we're putting enough effort into it to have maybe 90-95% uptime. It might be working more often than that, but if so we're getting lucky. If it happens to go down the day you need it for the course, we might not be able to notice and get it back up for you that day.

We also haven't tested more than a couple people banging on the demo server at a time. It only has a couple gigabytes of memory and it will break if enough simultaneous requests are made, and we have done no testing to find out what that number is. So if you're going to send an appreciably large class to try and use it all at once, you might need to do that testing yourself first.

If you get your computer's IP (for example, 10.1.0.226), you can run a tube map instance that others on your network can access (assuming you approve the necessary firewall holes) with:

HOST=10.1.0.226 npm run start

While that's running, anyone on the LAN can go to http://10.1.0.226:3001 and use your dev server. I'm sure React will tell you not to do this unless you trust the people on the LAN, though; I'm not sure what kind of exciting access the dev server gives them to your machine if you just open it up. I feel like there's some concept of a "PIN" for accessing the dev server's dangerous aspects, and I also think there might be a big security hole where it's not actually checked in the most recent version of the dev server available to the tube map through the now-dead react-scripts dependency that we haven't had the resources to migrate away from.

You can set up a safer non-dev server with npm run build && npm run serve, but that will only listen locally unless you set a serverBindAddress key in src/config.json first; we unfortunately don't have the host to use for that bound out to an environment variable.

[SimonaSecomandi]

Hi Adam, thank you so much, now it's working!!

The course will be abroad (on university computers) while I'm the US so it would not be the easiest thing to manage the tube map instance from my computer. I could try to let them set up sequenceTubeMap on each of the computers in the room, but I guess it would be too complicated. Now that the demo is working they will definitely be able to see the graph autonomously after the course anyway. Thanks for all the suggestion!!

[SimonaSecomandi]

Hi Adam, unfortunately, the demo is again not working :(

[adamnovak]

As of now it appears to be up; I can load the initial demo visualization.

Here's a screenshot of the AWS metrics for the host for the hour you posted; all the lines look properly wavy, so it doesn't seem to have been doing the thing it does where it drops off the network until the cron job we added to put it back on the network fixes it, or at least it didn't do it for more than a few minutes.

Do you mean it has somehow had a regression where it no longer works with your particular files, after previously working with those files, and is back to saying input path bTaeGut7_mat#0#chr22 not found in xg index? Or is it "not working" in some other way?

[adamnovak]

I've applied a second possible workaround for the dropping-network-connectivity issue, in case that's what was causing the problem here after all.

[SimonaSecomandi]

Hi Adam, the demo is still not working, meaning it can't even display the example graphs. Thanks you for the support!

[adamnovak]

I checked just now and it is working for me, without me touching anything. Is it also up for you now, or do you see it as down while I see it as up?