RFC: Provider Caching

[coffee-cup]

Providers should be able to specify what files are necessary for the install step to allow for better Docker layer caching.

For example, the NPM provider could specify that package.json is all that is necessary to install. If that files does not change between deploys, a cached layer can be used.

Provider caching can be split up into a few different types

1. Specify the files necessary to install so that only these files are keyed against when computing a cache hit/miss
2. App/framework level caches (e.g. .next directory). These caches should be re-used even if the source code that generated the cache changes. For this level of caching we cannot take advantage of built in Docker layer caching and will likely require a persistent volume that can be shared between builds.

This RFC will mainly focus on the first type.

[Milo123459]

I'd love to work on this, I have some ideas about how this can be made extremely modular.

[coffee-cup]

I was initially thinking of allowing providers to specify what files are necessary for the install step. This would allow us to take advantage of the Docker layer cache if these files do not change between builds.

fn install_files(&self, _app: &App) -> Result<Vec<String>> {
  Ok(Vec::new())
}

Alternatively install_cmd could be extended to return a list of (command, files) pairs, but ideally if the provider does not specify any install files, all files are presumed to be needed.

For example, in the NodeJS case, only the package.json file is necessary to run npm install. If the NPM provider was extended to

impl Provider for NpmProvider {
    // ...

    fn install_files(&self, _app: &App) -> Result<Vec<String>> {
      Ok(vec!["package.json".to_string()])
    }

    fn install_cmd(&self, _app: &App) -> Result<Option<String>> {
        Ok(Some("npm install".to_string()))
    }

   // ...
}

Then when creating the Dockerfile we can first only copy over the install files (package.json) file, run the install step (npm install), and then copy over the remaining files. If only the source code changes between builds then the dependencies installed with npm can be pulled from the cache.

Obviously, there are more advanced use cases with different languages and frameworks. Please share them here or bring up any concerns you see with this approach.

[Milo123459]

How would this handle for example the .next folder with NextJS? That isn't needed at install time but isn't (but can be used) at build time.

PS: This approach shouldn't error if the file doesn't exist.

[coffee-cup]

Hmm at the moment it wouldn't handle that case and caching for specific frameworks (next, remix, etc) will require a bit more involved solution. Happy to hear thoughts/suggestions here.

[coffee-cup]

I do think this should happen in multiple steps. First step to just get deps cached between builds (easy win). Next step can be to give providers and API for more advanced fine grain cache control.

[JakeCooper]

Hmm at the moment it wouldn't handle that case and caching for specific frameworks (next, remix, etc) will require a bit more involved solution. Happy to hear thoughts/suggestions here.

I kinda struggled with this a tiny bit when I took a swing at the Rust and Zig providers

I think we need a way to have postbuild actions which allow you to insert 0 - N "COPY" calls into the dockerfile to move them into the right place

[Milo123459]

I think we need a way to have postbuild actions which allow you to insert 0 - N "COPY" calls into the dockerfile to move them into the right place

I really like the idea of doing it like this. Not sure if COPY is the write call, more should be ADD but can be disclosed later.

[coffee-cup]

The providers sort of have that ability now. They can define files that are necessary for each phase and only those files are copied over when running that phase so we can make ues of the build cache. However it doesn't work all the time and there are definitely some holes and I would like to see it fleshed out some more.

Honestly I think we will need some semi-custom caching solution in the not too distant future because relying on the Docker layer cache isn't great and it would be nice to have more control over what is cached.

[Milo123459]

Where would cached files be stored if we don't use Docker caching? IMO, just using Docker caching is a better idea.

[coffee-cup]

Where would cached files be stored if we don't use Docker caching?

That is what we have to figure out 😆

The problem with Docker caching is that it is linear. A changed env var before the build phase will break the cache for every command below (which is fairly common on Railway because we pass in the github commit sha to every build). It would also be nice to benefit from the cache even if the files change. For example you can still benefit from the .next cache even if the source files you are building from change.

We will see though. Can optimize for the Docker cache now and see where the issues are and if/what we can do to improve it.