Description
Hey!
We are using a Google Cloud Storage Bucket through sccache
to cache across CI compilations of LLVM. Unfortunately, it looks like none of the cloud provider implementations allow for keeping the size of the cache in check — it is only possible for local-disk caching (which we can not do as the CI machines are ephemeral VMs).
In order to keep the costs of this shared cache reasonable, we implemented an Age lifecycle rule in GCS: if a cached entry is too old, GCS will deal with deleting it.
According to GCS, an Age rule is matched as:
Age is counted from when an object was uploaded to the current bucket.
This presents a problem with "death-waves": Every
It would be great if sccache
with a cloud bucket could behave in the same LRU way as ccache
locally does. Doing a full LRU is problematic without handling size limits, but it seems there is another lifecycle property that can be used, at least in the case of GCS: CustomTime.
LRU-like behaviour could be simulated by setting the Days since custom time lifecycle rule (instead of Age), making sure only files that had not been cache hit for
However, sccache
does not populate this field at all:
It seems from the documentation that this is a simple timestamp field that could be populated through the API, so all we need is a simple additional HTTP request that tells Google to update this field to "current time" once the caching logic hits a file successfully.
It seems like similar metadata options are available for other cloud providers, such as S3 or Azure but I have no idea or experience whether these could be used for controlling the cache's lifecycle as well as it can be for GCS.