docs: Incorporate RocketChip JTAG + GDB debugging instructions

Import detailed JTAG + GDB debugging instructions from the RocketChip README into Chipyard's documentation. This improves the Chip-Communication section with guidance for debugging RISC-V programs using the Remote Bit-Bang interface, OpenOCD, and GDB.

Documentation was sourced from this archive of the RocketChip README:
https://github.com/chipsalliance/rocket-chip/blob/0b7fef44f82b35bf289949b3f8fdc139641f8e10/README.md

Author: daniellovell

docs/Advanced-Concepts/Chip-Communication.rst

@@ -129,35 +129,250 @@ The default Chipyard designs instantiate the DTM configured to use JTAG (i.e. ``
     However, they also use TSI/Serialized-TL with FESVR in case the JTAG interface isn't used.
     This allows users to choose how to communicate with the DUT (use TSI or JTAG).
 
-Debugging with JTAG
-~~~~~~~~~~~~~~~~~~~
+Debugging with JTAG + GDB
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Roughly the steps to debug with JTAG in simulation are as follows:
+This section provides detailed instructions for using GNU debugger (GDB) to debug RISC-V programs running on the emulator, similar to debugging with Spike.
 
-1. Build a Chipyard JTAG-enabled RTL design. Remember default Chipyard designs are JTAG ready.
+Generating the Remote Bit-Bang (RBB) Emulator
++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The objective of this section is to use GNU debugger to debug RISC-V programs running on the emulator in
+the same fashion as in `Spike <https://github.com/riscv/riscv-isa-sim#debugging-with-gdb>`_.
+
+For that we need to add a Remote Bit-Bang client to the emulator. We can do so by extending our Config
+with ``JtagDTMSystem``, which will add a ``DebugTransportModuleJTAG`` to the DUT and connect a ``SimJTAG``
+module in the Test Harness. This will allow OpenOCD to interface with the emulator, and GDB can interface
+with OpenOCD. In the following example we add this Config alteration to
+``src/main/scala/system/Configs.scala``:
+
+.. code-block:: scala
+
+    class DefaultConfigRBB extends Config(
+      new WithJtagDTMSystem ++ new WithNBigCores(1) ++ new WithCoherentBusTopology ++ new BaseConfig)
+
+    class QuadCoreConfigRBB extends Config(
+      new WithJtagDTMSystem ++ new WithNBigCores(4) ++ new WithCoherentBusTopology ++ new BaseConfig)
+
+To build the emulator with ``DefaultConfigRBB`` configuration we use the command:
 
 .. code-block:: bash
 
-    cd sims/verilator
-    # or
-    cd sims/vcs
+    rocket-chip$ cd emulator
+    emulator$ CONFIG=freechips.rocketchip.system.DefaultConfigRBB make
+
+We can also build a debug version capable of generating VCD waveforms using the command:
+
+.. code-block:: bash
+
+    emulator$ CONFIG=freechips.rocketchip.system.DefaultConfigRBB make debug
+
+By default the emulator is generated under the name
+``emulator-freechips.rocketchip.system-DefaultConfigRBB`` in the first case and
+``emulator-freechips.rocketchip.system-DefaultConfigRBB-debug`` in the second.
+
+Compiling and executing a custom program using the emulator
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+We suppose that ``helloworld`` is our program. You can use ``crt.S``, ``syscalls.c``, and the linker script
+``test.ld`` to construct your own program (see
+`riscv-tests <https://github.com/riscv/riscv-tests>`_). Note that ``test.ld`` loads the program at
+``0x80000000`` so you will need ``-mcmodel=medany`` (see
+`The RISC-V Code Models <https://www.sifive.com/blog/2017/09/11/all-aboard-part-4-risc-v-code-models/>`_).
+
+.. code-block:: c
+
+    char text[] = "Vafgehpgvba frgf jnag gb or serr!";
 
-    make CONFIG=RocketConfig
+    // Don't use the stack, because sp isn't set up.
+    volatile int wait = 1;
 
-2. Run the simulation with remote bit-bang enabled. Since we hope to load/run the binary using JTAG,
-   we can pass ``none`` as a binary (prevents FESVR from loading the program). (Adapted from: https://github.com/chipsalliance/rocket-chip#3-launch-the-emulator)
+    int main()
+    {
+        while (wait)
+            ;
+
+        // Doesn't actually go on the stack, because there are lots of GPRs.
+        int i = 0;
+        while (text[i]) {
+            char lower = text[i] | 32;
+            if (lower >= 'a' && lower <= 'm')
+                text[i] += 13;
+            else if (lower > 'm' && lower <= 'z')
+                text[i] -= 13;
+            i++;
+        }
+
+        while (!wait)
+            ;
+    }
+
+First, we can test if this program executes correctly in the simpler emulator (non-RBB) before debugging:
 
 .. code-block:: bash
 
-    # note: this uses Chipyard make invocation to run the simulation to properly wrap the simulation args
-    make CONFIG=RocketConfig BINARY=none SIM_FLAGS="+jtag_rbb_enable=1 --rbb-port=9823" run-binary
+    $ ./emulator-freechips.rocketchip.system-DefaultConfig helloworld
 
-3. `Follow the instructions here to connect to the simulation using OpenOCD + GDB. <https://github.com/chipsalliance/rocket-chip#4-launch-openocd>`__
+Additional verbose information (clock cycle, pc, instruction being executed) can be printed:
+
+.. code-block:: bash
+
+    $ ./emulator-freechips.rocketchip.system-DefaultConfig +verbose helloworld 2>&1 | spike-dasm
+
+VCD output files can be obtained using the ``-debug`` version of the emulator and are specified using ``-v`` or
+``--vcd=FILE`` arguments. A detailed log file of all executed instructions can also be obtained from the emulator:
+
+.. code-block:: bash
+
+    $ ./emulator-freechips.rocketchip.system-DefaultConfig-debug +verbose -v output.vcd  helloworld 2>&1 | spike-dasm > output.log
+
+.. note::
+    Generated VCD waveforms and execution log files can be very voluminous depending on the size of the .elf file
+    (i.e. code size + debugging symbols).
 
 .. note::
-    This section was adapted from the instruction in Rocket Chip and riscv-isa-sim. For more information refer
-    to that documentation: `Rocket Chip GDB Docs <https://github.com/chipsalliance/rocket-chip#-debugging-with-gdb>`__,
-    `riscv-isa-sim GDB Docs <https://github.com/riscv/riscv-isa-sim#debugging-with-gdb>`__
+    The time it takes the emulator to load your program depends on executable size. Stripping the .elf executable
+    will unsurprisingly make it run faster. For this you can use ``$RISCV/bin/riscv64-unknown-elf-strip`` tool to
+    reduce the size. This is good for accelerating your simulation but not for debugging.
+
+.. warning::
+    The HTIF communication interface between our system and the emulator relies on ``tohost`` and ``fromhost``
+    symbols to communicate. If you try to run a totally stripped executable on the emulator, you may get:
+
+    .. code-block:: text
+
+        $ ./emulator-freechips.rocketchip.system-DefaultConfig totally-stripped-helloworld 
+        This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
+        Listening on port 46529
+        warning: tohost and fromhost symbols not in ELF; can't communicate with target
+
+    To resolve this, we need to strip all the .elf executable but keep ``tohost`` and ``fromhost`` symbols:
+
+    .. code-block:: bash
+
+        $ riscv64-unknown-elf-strip -s -Kfromhost -Ktohost helloworld
+
+    More details on the GNU strip tool can be found
+    `here <https://www.thegeekstuff.com/2012/09/strip-command-examples/>`_.
+
+The interest of this step is to make sure your program executes well. To perform debugging you need the original
+unstripped version, as explained in the following steps.
+
+Launch the emulator
++++++++++++++++++++++++++
+
+First, do not forget to compile your program with ``-g -Og`` flags to provide debugging support.
+
+We can then launch the Remote Bit-Bang enabled emulator with:
+
+.. code-block:: bash
+
+    ./emulator-freechips.rocketchip.system-DefaultConfigRBB +jtag_rbb_enable=1 --rbb-port=9823 helloworld
+
+.. note::
+    You can also use the ``emulator-freechips.rocketchip.system-DefaultConfigRBB-debug`` version instead if you
+    would like to generate VCD waveforms.
+
+.. note::
+    If the argument ``--rbb-port`` is not passed, a default free TCP port on your computer will be chosen randomly.
+
+.. note::
+    When debugging with GDB, the .elf file is not actually loaded by the FESVR. In contrast with Spike, it must
+    be loaded from GDB as explained in step 5. So the ``helloworld`` argument may be replaced by any dummy name.
+
+Launch OpenOCD
+++++++++++++++++++++
+
+You will need a RISC-V Enabled OpenOCD binary. This is installed with rocket-tools in ``$(RISCV)/bin/openocd``,
+or can be compiled manually from riscv-openocd. OpenOCD requires a configuration file, in which we define the RBB
+port we will use, which is in our case ``9823``.
+
+.. code-block:: tcl
+
+    interface remote_bitbang
+    remote_bitbang_host localhost
+    remote_bitbang_port 9823
+
+    set _CHIPNAME riscv
+    jtag newtap $_CHIPNAME cpu -irlen 5
+
+    set _TARGETNAME $_CHIPNAME.cpu
+    target create $_TARGETNAME riscv -chain-position $_TARGETNAME
+
+    gdb_report_data_abort enable
+
+    init
+    halt
+
+Then we launch OpenOCD in another terminal using the command:
+
+.. code-block:: bash
+
+    $(RISCV)/bin/openocd -f ./cemulator.cfg
+
+.. note::
+    A ``-d`` flag can be added to the command to show further debug information.
+
+Launch GDB
++++++++++++++++++
+
+In another terminal launch GDB and point to the elf file you would like to load then run it with the debugger
+(in this example, ``helloworld``):
+
+.. code-block:: bash
+
+    $ riscv64-unknown-elf-gdb helloworld
+
+Compared to Spike, the C Emulator is very slow, so several problems may be encountered due to timeouts between
+issuing commands and response from the emulator. To solve this problem, we increase the timeout with the GDB
+``set remotetimeout`` command.
+
+After that we load our program by performing a ``load`` command. This automatically sets the ``$PC`` to the
+``_start`` symbol in our .elf file:
+
+.. code-block:: none
+
+    (gdb) set remotetimeout 2000
+    (gdb) target remote localhost:3333
+    Remote debugging using localhost:3333
+    0x0000000000010050 in ?? ()
+    (gdb) load
+    Loading section .text.init, size 0x2cc lma 0x80000000
+    Loading section .tohost, size 0x48 lma 0x80001000
+    Loading section .text, size 0x98c lma 0x80001048
+    Loading section .rodata, size 0x158 lma 0x800019d4
+    Loading section .rodata.str1.8, size 0x20 lma 0x80001b30
+    Loading section .data, size 0x22 lma 0x80001b50
+    Loading section .sdata, size 0x4 lma 0x80001b74
+    Start address 0x80000000, load size 3646
+    Transfer rate: 40 bytes/sec, 520 bytes/write.
+
+Now we can proceed as with Spike, debugging works in a similar way:
+
+.. code-block:: none
+
+    (gdb) print wait
+    $1 = 1
+    (gdb) print wait=0
+    $2 = 0
+    (gdb) print text
+    $3 = "Vafgehpgvba frgf jnag gb or serr!"
+    (gdb) c
+    Continuing.
+
+    ^C
+    Program received signal SIGINT, Interrupt.
+    main (argc=0, argv=<optimized out>) at src/main.c:33
+    33	    while (!wait)
+    (gdb) print wait
+    $4 = 0
+    (gdb) print text
+    $5 = "Instruction sets want to be free!"
+
+For more information on GDB debugging, refer to:
+
+* `GDB User Manual <https://sourceware.org/gdb/onlinedocs/gdb/>`_
+* `GDB Remote Debugging <https://sourceware.org/gdb/onlinedocs/gdb/Remote-Debugging.html#Remote-Debugging>`_
 
 Example Test Chip Bringup Communication
 ---------------------------------------