Part 2: Update of doxygen/sphinx documentation for LimeSuiteNG library public API.

.github/workflows/documentation.yml

@@ -27,11 +27,8 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -qq doxygen graphviz
-
-      - name: Build doxygen
-        run: |
-          cd ${{github.workspace}}/docs/doxygen
-          doxygen DoxygenConfigLocal.in
+          cd ${{github.workspace}}
+          sudo ./install_dependencies.sh
 
       - name: Set up Python venv
         run: |
@@ -44,7 +41,7 @@ jobs:
         run: |
           cd ${{github.workspace}}/docs
           source venv/bin/activate
-          ./generate_docs.sh
+          ./generate_docs.sh rebuild
 
       # - name: Test hyperlinks
       #   run: |

docs/conf.py

@@ -211,7 +211,7 @@ def _read_local(path: Path) -> str:
 ## Extensions updated also
 
 breathe_projects = {
-    "Lime Suite NG" : ( "../docs/doxygen/output/xml" )
+    "Lime Suite NG" : ( "../build/docs/doxygen/xml" )
 }
 
 breathe_default_project = "Lime Suite NG"

docs/development/documentation.rst

@@ -2,33 +2,43 @@ Documentation generation guide
 ==============================
 
 This page describes how to generate the LimeSuiteNG documentation
-(the very thing you're reading right now) on Linux.
+(the very thing you're reading right now) on Linux or Windows platforms.
 
 Prerequisites
 -------------
 
 Components needed to generate the LimeSuiteNG documentation:
 
 - `GCC`_
+- `Ninja`_ (For Windows)
 - `CMake`_
-- `Python 3.8`_
+- `Python (>=3.8)`_
 - `Doxygen`_
+- `Graphviz`_
 
 Setup
 -----
 
 To set up the environment for the documentation generation, you will need to set up a `Python virtual environment (venv)`_.
 
-To set up the environment, run these commands:
+To set up the environment on Linux, run these commands:
 
 .. code-block:: bash
 
-    python3.8 -m venv venv # Create the virtual environment directory
+    python -m venv venv # Create the virtual environment directory
     source venv/bin/activate # Activate the virtual environment
     pip install -r requirements.txt # Install all the required dependencies for the generation
 
+To set up the environment on Windows, run these commands:
+
+.. code-block:: doscon
+
+    python -m venv venv # Create the virtual environment directory
+    venv\Scripts\activate # Activate the virtual environment
+    pip install -r requirements.txt # Install all the required dependencies for the generation
+
 .. note::
-    One should run these commands in the ``docs`` directory.
+    One should run these commands in the ``docs`` directory of cloned repository.
 
 Generation
 ----------
@@ -37,23 +47,53 @@ In the ``docs`` directory, located in the root directory of the repository, whil
 
 .. code-block:: bash
 
-    cmake -S .. -B ../build # Generate the make file for the suite.
-    make --no-print-directory -C ../build doc # Build Doxygen documentation
-    breathe-apidoc --generate class --members --force --output-dir apidoc ../build/xml/ # Generate the class API pages
-    python add_undoc_members.py # Add a flag to add all undocumented members into the page
-    make html # Generate the documentation itself
+    cmake -S .. -B ../build # 1. Generate the configuration files for the suite. For Windows include: -G Ninja
+    cmake --build ../build -- doxygen # 2. Run only the Doxygen target to build doxygen
+
+    # 3. Generate reference pages for API
+    breathe-apidoc --generate class --members --force --output-dir doxygen/api_member_list ../build/docs/doxygen/xml/
+    breathe-apidoc --generate file --force --output-dir doxygen/api_member_list ../build/docs/doxygen/xml/
+    breathe-apidoc --generate struct --members --force --output-dir doxygen/api_member_list ../build/docs/doxygen/xml/
+
+    # 4. Remove redundant copies of the actual manual pages
+    rm doxygen/api_member_list/file/*dox.rst    # use `del` instead of `rm` on Windows 
+
+    # 5. Regenerate .rst manual pages from .dox manual pages
+    python dox_converter.py
+
+    # 6. Generate the final documentation
+    make html 
+
+.. tip::
+
+    When running documentation build commands on Windows, do not forget to swap all instances of forward slash ``/`` with backslash ``\``.
 
 .. important::
     These commands or the script must be run in the venv, otherwise it will fail.
 
-For ease of convenience, there is also a ``generate_docs.sh`` script located in the ``docs`` directory.
+For ease of convenience, ``docs`` directory contains scripts that automate the documentation build process for each platform.
+On Linux, run ``generate_docs.sh`` script. On Windows, run ``generate_docs.bat`` script. Scripts support additional command line arguments:
+
+- ``clean`` - Deletes all documentation build artifacts.
+- ``rebuild`` - Runs the documentation build steps.
+
+To simultaneously clean and rebuild the documentation, pass the arguments in the following order - ``generate_docs.sh clean rebuild``. 
+Running script without any command line arguments, will trigger only the sphinx documentation build tool.
+
+.. note::
+
+    Scripts ``generate_docs.sh`` and ``generate_docs.bat`` must be run from ``docs`` directory.
 
 After a successful generation the resulting documentation pages will be located in
 ``docs/_build/html`` directory.
 
 Link checking
 -------------
 
+.. note::
+
+    For Linux only.
+
 To check whether all URLs in the documentation are valid, there exists a ``make`` target to automatically check the links:
 
 .. code-block:: bash
@@ -69,6 +109,10 @@ Any broken links will be shown in red and also listed in the ``_build/linkcheck/
 Docker
 ------
 
+.. note::
+
+    For Linux only.
+
 There also exists a Docker file to generate the documentation.
 To generate this documentation using Docker, in the root directory of the repository run:
 
@@ -85,8 +129,10 @@ For more information about how to set up and write the documentation,
 check out the `MyriadRF Handbook`_.
 
 .. _`GCC`: https://gcc.gnu.org/
+.. _`Ninja`: https://ninja-build.org/
 .. _`CMake`: https://cmake.org/
-.. _`Python 3.8`: https://www.python.org/downloads/release/python-3818/
+.. _`Python (>=3.8)`: https://www.python.org/downloads/release/python-3818/
 .. _`Doxygen`: https://www.doxygen.nl/
+.. _`Graphviz`: https://graphviz.org/
 .. _`Python virtual environment (venv)`: https://docs.python.org/3.8/library/venv.html
 .. _`MyriadRF Handbook`: https://handbook.myriadrf.org/

docs/dox_converter.py

@@ -0,0 +1,137 @@
+"""
+Doxygen to Sphinx page converter
+================================
+
+Python module performs quick auto parsing of the regular doxygen pages (with @page tag) and generates 
+the equivalent sphinx page files (.rst files) with the required Sphinx Breathe plugin directives. This 
+module can be run in reverse mode to delete all the generated manual pages (.rst files). See supported 
+flags for more info.
+
+Usage:
+   ``python dox_converter.py [override-root-path] [optional-flags]``
+
+This module will auto scan each subdirectory starting from the root directory (by default that is ``./doxygen``) and will
+generate .rst files equivalent to all the .dox files. To override the root directory, pass the root directory as a command 
+line argument when calling the module.
+
+Supported flags:
+
+* `--del` - Runs dox_converter in reverse. Deletes all the .rst manual pages that were converted from doxygen manual pages.
+
+This module does not support the auto generation of index files used to organize navigation between pages.
+Users are responsible for setting up the correct index tree. Module will ignore files that have the following 
+names: ``mainpage.dox, index.dox``. Therefore, it is recommended to use these file names for doxygen index pages. 
+If index page naming is ignored, the module can produce undesired results.
+
+"""
+
+from pathlib import Path
+import sys
+
+reverse = False                  # Dox converter in reverse mode: deletes all of the generated manual pages in .rst format.
+root_path = Path("./doxygen/")   # Default root path
+dox_suffix = ".dox"              # Doxygen suffix indentifier
+rst_suffix = ".rst"              # Sphinx file suffix indentifier
+dir_tree = []                    # The directory tree view from the root path 
+
+index_indicators = ["mainpage", "index"]        # File names that indicate index file with toc tree
+ignore_dirs = ["output", "api_member_list", "images"]   # Directories to ignore
+
+
+def update_dir_tree(start_dir, ignore_list = ignore_dirs):
+   """ 
+   Creates a directory tree starting from the root directory.
+
+   Starting from the start_dir, recursively scans through the directories and adds 
+   them to the tree view, which is stored in dir_tree list.
+
+   """
+   dir_tree.append(start_dir)
+   for dir_item in start_dir.iterdir():
+      if dir_item.is_dir():
+
+         # skip subdir if it is in ignore list
+         if dir_item.name in ignore_list:
+            continue
+         update_dir_tree(dir_item)
+      else:
+         continue
+
+def get_files(start_dir, suffix):
+   """Gets the list of all files in specified diretory."""
+
+   files = []
+   for dir_item in start_dir.iterdir():
+      if dir_item.is_file() and dir_item.suffix == suffix:
+         files.append(dir_item)
+      else:
+         continue
+
+   return files
+
+def convert_page(file):
+   """Parses doxygen page title line elements and applies rst page template with breathe directives."""
+
+   rst_path = file.with_suffix(".rst")
+   rst = rst_path.open("w")
+   with file.open("r") as dox:
+      dox.readline()
+      header = dox.readline()
+   header = header.strip()
+   header_elements = header.split(maxsplit=3)
+   rst.write(f"{header_elements[3]}\n{"=" * len(header_elements[3])}\n\n")
+   rst.write(f".. doxygenpage:: {header_elements[2]}\n\t:content-only:")
+   rst.close()
+
+def convert_pages(files):
+   """Generates sphinx compatible rst manual pages with breathe directives from doxygen pages."""
+
+   for file in files:
+      if file.stem not in index_indicators:
+         convert_page(file)
+
+def remove_pages(files):
+   """Removes sphinx compatible rst manual pages."""
+
+   for file in files:
+      if file.stem not in index_indicators:
+         file.unlink()
+
+def print_tree():
+   """Prints directory tree with contents of each directory."""
+
+   for dir in dir_tree:
+      print(f"{dir} contents:")
+      for item in dir.iterdir():
+         print(f"\t{item.name}")
+      print("\n")
+
+
+########################################
+#  Script start
+########################################
+
+if len(sys.argv) > 1:
+   for arg in sys.argv[1:]:
+      if arg == "--del":
+         reverse = True
+      else:
+         root_path = Path(f"{arg}")
+         print(f"Doxygen page converter: Switching to new root path - {arg}")
+
+update_dir_tree(root_path)
+# print_tree()
+
+# Delete generated rst manual pages
+if reverse:
+   for dir in dir_tree:
+      files = get_files(dir, rst_suffix)
+      remove_pages(files)
+
+# Generate regular rst pages from regular doxygen pages
+else:
+   for dir in dir_tree:
+      files = get_files(dir, dox_suffix)
+      convert_pages(files)
+
+# print_tree()

docs/doxygen/DoxygenConfig.in

@@ -17,6 +17,8 @@
 # Project related configuration options
 #---------------------------------------------------------------------------
 
+# For project flags, go to ENABALED_SECTIONS tag.
+
 # This tag specifies the encoding used for all characters in the configuration
 # file that follow. The default is UTF-8 which is also the encoding used for all
 # text before the first occurrence of this tag. Doxygen uses libiconv (or the
@@ -269,7 +271,8 @@ TAB_SIZE               = 4
 # commands \{ and \} for these it is advised to use the version @{ and @} or use
 # a double escape (\\{ and \\})
 
-ALIASES                =
+ALIASES                = "rstVerbatim=@verbatim embed:rst:leading-asterisk"
+ALIASES               += "embedRstVerbatim{1}=@verbatim embed:rst:leading-asterisk \1 ^^ @endverbatim"
 
 # Set the OPTIMIZE_OUTPUT_FOR_C tag to YES if your project consists of C sources
 # only. Doxygen will then generate output that is more tailored for C. For
@@ -717,8 +720,16 @@ GENERATE_DEPRECATEDLIST= YES
 # The ENABLED_SECTIONS tag can be used to enable conditional documentation
 # sections, marked by \if <section_label> ... \endif and \cond <section_label>
 # ... \endcond blocks.
+# List of custom Doxygen configuration flags:
+# RST_SUPPORT - enable swaping of doxygen sections and special commands with RST markup style elements. This flag is used for compatibility with breathe plugin, which translates doxygen output 
+#               to RST markup style when runing Sphinx documentation build. Use this only when the final doxygen output is included in sphinx documentation build.
+# DEVELOP     - enables the output of incomplete doxygen sections and other development marks in the final doxygen output. Remove this flag to hide incomplete doxygen sections or other 
+#               development marks in the final doxygen output.
+# DOX_NAV     - enable page references for navigation purposes in the final doxygen output.
 
-ENABLED_SECTIONS       =
+ENABLED_SECTIONS       = RST_SUPPORT
+#ENABLED_SECTIONS       += DEVELOP
+#ENABLED_SECTIONS       += DOX_NAV
 
 # The MAX_INITIALIZER_LINES tag determines the maximum number of lines that the
 # initial value of a variable or macro / define can have for it to appear in the
@@ -868,6 +879,7 @@ WARN_LOGFILE           =
 # Specified INPUT path are relative to docs/doxygen directory.
 
 INPUT                  = @PROJECT_SOURCE_DIR@/src/include @PROJECT_SOURCE_DIR@/src/comms/SPI/ISPI.h @PROJECT_SOURCE_DIR@/src/comms/II2C.h
+INPUT                 += @PROJECT_SOURCE_DIR@/docs/doxygen
 
 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
@@ -907,11 +919,11 @@ FILE_PATTERNS          = *.c \
                          *.hpp \
                          *.h++ \
                          *.inc \
+                         *.dox
                          #*.m \
                          #*.markdown \
                          # *.md \
                          #*.mm \
-                         #*.dox
 
 # The RECURSIVE tag can be used to specify whether or not subdirectories should
 # be searched for input files as well.
@@ -1191,7 +1203,7 @@ IGNORE_PREFIX          =
 # If the GENERATE_HTML tag is set to YES, doxygen will generate HTML output
 # The default value is: YES.
 
-GENERATE_HTML          = YES
+GENERATE_HTML          = NO
 
 # The HTML_OUTPUT tag is used to specify where the HTML docs will be put. If a
 # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of
@@ -2072,7 +2084,7 @@ MAN_LINKS              = NO
 # captures the structure of the code including all documentation.
 # The default value is: NO.
 
-GENERATE_XML           = NO
+GENERATE_XML           = YES
 
 # The XML_OUTPUT tag is used to specify where the XML pages will be put. If a
 # relative path is entered the value of OUTPUT_DIRECTORY will be put in front of