Generate dependency map for sidebar, docusaurus.config, etc. in hot reload

[dsalaza4]

🐛 Bug Report

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io

Description

In order to avoid having a huge sidebar.js file,
we created a
main.js file and imported sidebars for different sections from external js files.

We then specified main.js as the sidebarPath.

The problem with this approach is:

1. Every time that an imported file changes, the dev server does not recognize the change, thus not updating the sidebar accordingly.
2. Even if the main.js file changes, it looks like it is caching the imported variables from the other files, which also prevents the update.

Have you read the Contributing Guidelines on issues?

Yes I have.

Steps to reproduce

1. Use a main.js file for your sidebar that imports sub-sidebars from other files like
   here.
2. Start the development server with docusaurus start.
3. Modify one of the imported files.
4. Notice how the dev server does not see the change.

Expected behavior

Docusaurus should be able to notice which files are being imported
by main.js and update its sidebar accordingly

Actual behavior

Docusaurus only updates if main.js changes,
and even if it does change,
imported variables from other files are not updated.

Your environment

• Public source code: https://gitlab.com/fluidattacks/product/-/tree/master/docs/src
• Public site URL: https://docs.fluidattacks.com/
• Docusaurus version used: 2.0.0-beta.0
• Environment name and version (e.g. Chrome 78.0.3904.108, Node.js 10.17.0): N/A
• Operating system and version (e.g. Ubuntu 20.04.2 LTS): N/A

Reproducible demo

https://codesandbox.io/s/competent-sun-t28ho?file=/sidebar/secondary.js

You can try:

1. Removing secondary/two from SECONDARY in sidebar/secondary.js.
2. Noticing how nothing changes.
3. Removing SECONDARY from main.js.
4. Noticing how the sidebar disappears.
5. Adding SECONDARY again to main.js.
6. Noticing how it adds the version that contains secondary/one and secondary/three
   instead of reloading the file and justadding secondary/one

[slorber]

Agree, the current code only watch for changes in the sidebar path to reload the docs plugin content.

We should walk the dependency graph and also watch the required files.

This lib looks useful but not very modern (does not support ESM, which we may need soon), https://github.com/browserify/module-deps

If you find anything better that we could use, let me know

[dsalaza4]

Would it be possible, as a first step, that instead of trying to automatically generate the graph, we could explicitly provide the files to the config?

That way we could trigger an update if one of the provided files changes.

What I still do not know how to fix is avoiding caching requires, as it does not really matter if the main.js files changes, it still uses cached versions of the imported variables 😕

[dacevedo12]

There seems to be another problem beyond the file watcher. Even if chokidar triggers the reload, the content doesn't change due to require being cached.

Shouldn't WDS take care of that by default?

[slorber]

The community answered on twitter with a few interesting tools to get the depenend files of your sidebar js module: https://twitter.com/NicoloRibaudo/status/1407722571722858501

Would it be possible, as a first step, that instead of trying to automatically generate the graph, we could explicitly provide the files to the config?

We don't have a very good API for that but you can "enhance" an existing plugin (see comments here: #4138 (comment))

You should be able to wrap the default docs getPathsToWatch lifecycle (https://docusaurus.io/docs/lifecycle-apis#getpathstowatch) to add the additional paths you want to the list.

What I still do not know how to fix is avoiding caching requires, as it does not really matter if the main.js files changes, it still uses cached versions of the imported variables 😕

We use this to avoid caching issues inside Docusaurus: https://github.com/sindresorhus/import-fresh, maybe it will work for you too

Shouldn't WDS take care of that by default?

No, it only hot reload frontend code, not nodejs code afaik