Improve Freeway performance reading content at /path where root directory CID is shared across many CARs #31
Description
Freeway doesn't do super well in general reading a ton of CAR shards. This will be better moving forward as we're able to store larger shards, but it's something we need to figure out for historical data (where performance here has the potential to get even worse as we move to Saturn, etc.).
The key use case to focus on here is when users look to read individual files within a large directory split across many CARs. Right now we have to read all the shards to get to where the file is in the directory. This definitely has the potential to improve (vs. rendering the root directory structure, which always needs to grab all the CAR shards).
One idea @vasco-santos had was to keep an index of the location of the individual files after the first, slower read, so every subsequent read is more performant. If we do choose to have such a data structure, it'd be good if it was implemented in a similar way to DUDEWHERE and SATNAV so we can utilize this in any future models for Saturn, etc. (e.g., public claim-based model).