Update only stale files

[ed2050]

pydoctor seems great, exactly what I'm looking for in doc generation. I just can't seem to find a way to update stale files without rebuilding entire project. Does this exist? If not, how can I add it?

On large projects, rebuilding entire doc tree when just a few source files have changed can be very inefficient and slow. It's better to only rebuild stale pages. I tried:

• command line options. none address this scenario
• config file options. same as command line.
• build with stale files explicitly passed (eg pydoctor -c project.ini stale1.py stale2.py). doesn't work, pydoctor rewrites all doc files in output dir. files like index.html, nameIndex.html, moduleIndex.html will be wrong, only contain stale files after rebuild.

I envision an option -u, --update that tells pydoctor to refresh only files given on command line. This would only output foo.html for each listed input file foo.py, skipping common files like *.css and index*.html.

What would it take to achieve this?

[tristanlatr]

Hello @ed2050 and thanks for taking the time to open this issue.

There is one option that might cover what you are looking for: it's the option --html-subject. This repeatable option takes the fullName of objects to generate API docs for. If you provide the name a module, only that module's HTML and all it's content will be generated. As a consequence of using that option for in incrementally re-generating part of a project's documentation:

• the index files (Modules/Classes/Names/Search) will be stale since they are NOT generated when using --html-subject and
• the intersphinx inventory will be overriden and will only contains objects in the given --html-subjects

You can work around these issues by re-calling again pydoctor with options:
--make-html --html-summary-pages --make-intersphinx

But still, this method has some limitations:

• If an object is re-exported and only it's implementation location is re-generated, that object will not have it's documentation updated since it lives in another location that has not been passed to --html-subject.
• Same kind of issue arise with inherited methods and their documentation, if a superclass changed, all the locations of it's subclasses must be re-generated to reflect the change.

I think an approach based on --html-subject might work for some simple changes that does not rely on class hierarchy or re-exporting feature. But a full rebuild is still necessary once in a while.

Tell me what you think,