Are API docs needed for this repo?

[yousefmoazzam]

There's API docs being generated in the sphinx-build.sh script:

httomo-backends/docs/sphinx-build.sh

Lines 27 to 28 in a34bf58

	# sphinx-apidoc generates source files that use sphinx.ext.autodoc to document all found modules
	sphinx-apidoc -feT -t=$DIR/source/_templates -o $DIR/source/api $DIR/../httomo_backends

and similarly in the CI:

httomo-backends/.github/workflows/httomo_backends_docs.yml

Lines 34 to 35 in a34bf58

	- name: Build api docs
	run: sphinx-apidoc -feT -t=./docs/source/_templates -o ./docs/source/api ./httomo_backends

but the output rst files from this aren't being included in index.rst (so the API docs aren't included in the final docs deployment).

So that's one reason why I'm questioning if API docs are needed for this repo, because they're being built but not being used in the final docs deployment.

Another reason is if this repo needs API docs. At the moment, there's a handful of scripts used for generating the YAML templates, and that's it. The httomo_backends/ dir isn't even a python package (there's no __init__.py file), so there's no python modules in this repo really, just some scripts for generating YAML templates.

One question is: do we need/want API docs for the handful of scripts currently here?

Another question: for stuff that gets moved to httomo-backends (such as memory estimators and padding calculators from httomo, see DiamondLightSource/httomo#429), would we like API docs for those? At the moment in httomo, memory estimators are private functions where they reside, but I imagine that when they get moved into httomo-backends they'll be made public functions - which may make sense to have API docs as they're public, but may also not make sense to have API docs, because they all have the same interface (parameters), and their usage is made obvious by how they are used in httomo. In which case, perhaps API docs in httomo-backends isn't really needed, but rather, general documentation on how the memory estimator and padding calculator functions are used in httomo/how they hook into httomo.

[dkazanc]

cheers, I don't think we need to build API docs for this repo at all. Even when the memory estimators are moved, they are simple enough to understand their purpose without documentation for each one of the function. They are also quite internal for httomo really, so no need to expose that in the docs?

[yousefmoazzam]

After a bit more investigation, things aren't as simple as they first appeared regarding the API docs rst files being generated as shown in the original post.

It looks like the rst files being generated from running sphinx-apidoc, while not being used for API docs, are actually being used by yaml_doc_generator.py to generate the docs that include the YAML template, via modifying the API docs rst files:

httomo-backends/docs/source/yaml_doc_generator.py

Lines 35 to 46 in a34bf58

	doc_rst_file = f"{doc_dir}/api/{rst_name}.rst"
	with open(doc_rst_file, "a") as edit_doc:
	if os.stat(doc_rst_file).st_size == 0:
	rst_name = root.split("/")[-1]
	add_title(edit_doc, rst_name)
	add_backend_link(edit_doc, rst_name)
	else:
	edit_doc.write(f"\n\n .. rubric:: **Functions**")
	edit_doc.write(f"\n\n .. autosummary::\n")
	for fi in files:
	yml_title = fi.split(".yaml")[0]
	edit_doc.write(f"\n {yml_title}")

So, it's like the rst files being generated by sphinx-apidoc are being used for purposes other than API docs. It's a bit strange, and this approach has some imperfections (such as sphinx-apidoc attempting to generate API docs rst files for everything else in the httomo_backends/ dir, which is not needed for the YAML templates), but I can see how it would work to produce the YAML templates docs.

I think it'd be better to generate the YAML templates docs independent of sphinx-apidoc, but that is a different problem to what this issue is for.

For now, I'd suggest leaving this as it is while we decide how exactly to handle this, and I'll make another issue describing the dependency of the YAML templates on sphinx-apidoc.