Skip to content

[Bug] Exponential growth of template processing time for large repositories due to shared RawModels #10506

Open
@banchan86

Description

@banchan86

Describe the bug
When using the templating system it is often desirable to set the isShared flag in the function exports.getOptions to true in order to access information from other objects to customize the API template. However, this leads to growing of the RawModel size, which becomes an issue especially for large repositories and leads to exponential growth of the applying template to X models part of the docfx build process.

To Reproduce
Steps to reproduce the behavior:

  1. Choose a large repository (few hundred - thousand API objects)
  2. Create a template folder with a file called 'ManagedReference.overwrite.js`.
exports.getOptions = function (model) {
    return {
        isShared: true
    };
}
  1. Add template folder to docfx.json template property.
  2. Run
dotnet docfx --exportRawModel  
  1. On a repository that produces ~1000 models, this takes 30 minutes on my computer and generates ~1000 individual RawModel files that are each ~70Mb in size and >1.8 million lines. Even if the RawModel files are not being exported, template processing is very slow, so I think the problem is just that the input data model is just very large which affects the exports.transform function for every item.

Expected Behaviour
While I think it is expected behaviour, I think it could be optimized.

Desired Solution
Is there a way to control what is being shared, such that the RawModel size is not so large? I see two possible options:

  1. Filtering which RawModels to share.
  2. Filtering which items in a RawModel are being shared.

This might already be doable with exports.getOptions, but I am not sure how, so if someone can point me to an example that would be great!

Context (please complete the following information):

  • Browser: Chrome
  • Form factor: Desktop
  • Docfx version: 2.78.2
  • Template: default, custom.

Metadata

Metadata

Assignees

No one assigned

    Labels

    templateThe stock site template

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions