Reword and correct the preprocessor cache mode documentation

docs/Local.md

@@ -9,41 +9,46 @@ The local storage only supports a single sccache server at a time. Multiple conc
 ## Preprocessor cache mode
 
 This is inspired by [ccache's direct mode](https://ccache.dev/manual/3.7.9.html#_the_direct_mode) and works roughly the same.
+It adds a cache that allows to skip preprocessing when compiling C/C++. This can make it much faster to return compilation results
+from cache since preprocessing is a major expense for these.
 
-It can be set with by setting the variable:
-```
-SCCACHE_DIRECT=true
-```
+Preprocessor cache mode is controlled by a configuration option which is true by default, as well as additional conditions described below.
 
-In preprocessor cache mode, sccache caches the preprocessor step for C/C++ whenever possible. This can make the compilation a lot faster, since the preprocessor accounts for a non-negligible amount of time in the entire compile chain.
+To ensure that the cached preprocessor results for a source file correspond to the un-preprocessed inputs, sccache needs
+to remember, among other things, all files included by the source file. sccache also needs to recognize
+when "external factors" may change the results, such as system time if the `__TIME__` macro is used
+in a source file. How conservative sccache is about some of these external factors is configurable, see below.
 
-In order to cache the preprocessor step sccache needs to remember, among other things, all files included by the given input file. To quote ccache's documentation:
+Preprocessor cache mode will be disabled in any of the following cases:
 
-> There is a catch with the [preprocessor cache] mode: header files that were used by the compiler are recorded, but header files that were not used, but would have been used if they existed, are not. So, when [sccache] checks if a result can be taken from the cache, it currently can’t check if the existence of a new header file should invalidate the result. In practice, the [preprocessor cache] mode is safe to use in the absolute majority of cases.
+- The configuration option is false
+- Using distributed compilation (provisional fix for )
+- Not using GCC or Clang with local storage (only implemented for these)
+- The modification time of one of the header files is too new (avoids a race condition)
+- A compiler option not supported by preprocessor cache mode is used
+- Certain strings such as `__TIME__` are present in the source code, indicating that the preprocessor result may change based on external factors
 
-Preprocessor cache mode will be disabled if any of the following holds:
+The preprocessor cache may silently produce stale results in any of the following cases:
 
-- the configuration setting `use_preprocessor_cache_mode` is false
-- a modification time of one of the include files is too new (needed to avoid a race condition)
-- a compiler option not supported by the preprocessor cache mode is used. Currently, this is only `-Xpreprocessor` and `-Wp,*`, but if/when sccache grows to handle options then more could be added to this list.
-- the string `__TIME__` is present in the source code
+- When a source file was compiled and its results were cached, a header file would have been included if it existed, but it did
+  not exist at the time. sccache does not know about such files, so it cannot invalidate the result if the header file later exists.
+- A macro such as `__TIME__` is used in the source code and `ignore_time_macros` is enabled
+- There are other external factors influencing the preprocessing result that sccache does not know about
 
 Configuration options and their default values:
 
-- `use_preprocessor_cache_mode`: `true`. Whether to use preprocessor cache mode entirely.
+- `use_preprocessor_cache_mode`: `true`. Whether to use preprocessor cache mode. This can be overridden for an sccache invocation by setting the environment variable `SCCACHE_DIRECT` to `true`/`on`/`1` or `false`/`off`/`0`).
 - `file_stat_matches`: `false`. If false, only compare header files by hashing their contents. If true, will use size + ctime + mtime to check whether a file has changed. See other flags below for more control over this behavior.
 - `use_ctime_for_stat`: `true`. If true, uses the ctime (file status change on UNIX, creation time on Windows) to check that a file has/hasn't changed. Can be useful to disable when backdating modification times in a controlled manner.
 
-- `ignore_time_macros`: `false`. If true, ignore `__DATE__`, `__TIME__` and `__TIMESTAMP__` being present in the source code. Will speed up preprocessor cache mode, but can result in false positives.
+- `ignore_time_macros`: `false`. If true, ignore `__DATE__`, `__TIME__` and `__TIMESTAMP__` being present in the source code. Will speed up preprocessor cache mode, but can produce stale results.
 
 - `skip_system_headers`: `false`. If true, preprocessor cache mode will not cache system headers, only add them to the hash.
 
-- `hash_working_directory`: `true`. If true, will add the current working directory in the hash to distinguish two compilations from different directories.
+- `hash_working_directory`: `true`. If true, will add the current working directory to the hash to distinguish two compilations from different directories.
 
 See where to write the config in [the configuration doc](Configuration.md).
 
-*Note that preprocessor caching is currently only implemented for GCC and Clang and when using local storage.*
-
 ## Read-only cache mode
 
 By default, the local cache operates in read/write mode. The `SCCACHE_LOCAL_RW_MODE` environment variable can be set to `READ_ONLY` (or `READ_WRITE`) to modify this behavior.

src/compiler/c.rs

@@ -410,6 +410,7 @@ where
         }
 
         let use_preprocessor_cache_mode = {
+            // Disable preprocessor cache when doing distributed compilation (+ other conditions)
             let can_use_preprocessor_cache_mode = !may_dist
                 && preprocessor_cache_mode_config.use_preprocessor_cache_mode
                 && !too_hard_for_preprocessor_cache_mode;
@@ -438,7 +439,6 @@ where
             use_preprocessor_cache_mode
         };
 
-        // Disable preprocessor cache when doing distributed compilation
         let mut preprocessor_key = if use_preprocessor_cache_mode {
             preprocessor_cache_entry_hash_key(
                 &executable_digest,