PMP: difference between load and store behaviour for size > XLEN

[Wren6991]

The PMP section says:

riscv-isa-manual/src/machine.adoc

Lines 3301 to 3307 in 98ea4b5

	On some implementations, misaligned loads, stores, and instruction
	fetches may also be decomposed into multiple accesses, some of which may
	succeed before an access-fault exception occurs. In particular, a
	portion of a misaligned store that passes the PMP check may become
	visible, even if another portion fails the PMP check. The same behavior
	may manifest for stores wider than XLEN bits (e.g., the FSD instruction
	in RV32D), even when the store address is naturally aligned.

Stores with width greater than XLEN are allowed to tear against PMP checks. I didn't realise until I re-read this today that there is no such allowance for loads with width greater than XLEN.

Given loads are used for memory-mapped IO, the lack of tearing on loads is an observable aspect of the core's behaviour. This seems to make it illegal to implement Zilsd with a 32-bit data bus if you allow ld to IO regions and you have a PMP grain G = 0. (Or at least, illegal to implement ld as lw + lw).

I'm hoping this is an oversight given Zilsd was specifically modified in places to make it easier to implement on 32-bit-native RV32 implementations, and the Zilsd spec explicitly gives permission for 32-bit tearing in a few places. Could somebody please clarify?

[ved-rivos]

The RVWMO memory model states that, for aligned memory instructions, exactly one memory operation is generated, with two exceptions:

Among instructions in RV32GC and RV64GC, each aligned memory instruction gives rise to exactly one
memory operation, with two exceptions. First, an unsuccessful SC instruction does not give rise to any
memory operations. Second, FLD and FSD instructions may each give rise to multiple memory operations
if XLEN < 64, as stated in Section 21.3 and clarified below. An aligned AMO gives rise to a single memory
operation that is both a load operation and a store operation simultaneously.

Section 21.3 states:

FLD and FSD are only guaranteed to execute atomically if the effective address is naturally aligned and
XLEN ≥ 64.

However, additional exceptions now exist due to Zcmp and Zilsd. It may be better for the RVWMO text to state this rule more generally—for example, in terms of any memory instruction that performs memory accesses larger than XLEN and is not an AMO. Likewise, the text in the PMP section could be generalized.

[Wren6991]

Thanks for the extra citations, those are definitely relevant and I wasn't aware of them. Just to get all the related info lined up in this issue, here are some parts of Zilsd I think are relevant:

riscv-isa-manual/src/zilsd.adoc

Lines 37 to 39 in 98ea4b5

	[#norm:Zilsd_align8_no_exc]#As the access size is 64-bit, accesses are only considered naturally aligned for effective addresses that are a multiple of 8.
	In this case, these instructions are guaranteed to not raise an address-misaligned exception.#
	[#norm:Zilsd_align8_atomic_unknown]#Even if naturally aligned, the memory access might not be performed atomically.#

(emphasis last part -- "the memory access might not be performed atomically" sounds like it might contradict or be an exception to the RVWMO rule).

riscv-isa-manual/src/zilsd.adoc

Lines 114 to 115 in 98ea4b5

	[#norm:Zilsd_RVWMO_exc_misaligned]#For the purposes of RVWMO and exception handling, LD and SD instructions are
	considered to be misaligned loads and stores#, with one additional constraint:

(this seems like an explicit exception to the RVWMO rule.)

[gfavor]

"the memory access might not be performed atomically" sounds like it might contradict or be an exception to the RVWMO rule).

"For the purposes of RVWMO and exception handling, LD and SD instructions are considered to be misaligned loads and stores#, with one additional constraint" seems like an explicit exception to the RVWMO rule.

Yes, the preceding two text excerpts are calling out a similar exception to RVWMO that is introduced by Zilsd, i.e. that 64-bit accesses by Zilsd LD/SD instructions are not considered to be aligned accesses and hence are not guaranteed to be atomic.

Since the RVWMO spec already tries to call out all exception cases, it is appropriate to expand that list in the way Ved describes.

As far as the PMP spec, there doesn't really seem to be a need to modify it since it says that "On some implementations, misaligned loads, stores, and instruction fetches may also be decomposed into multiple accesses, some of which may succeed before an access-fault exception occurs." So as long as Zilsd says that its LD/SD instructions are considered to be misaligned loads/stores, then the rest falls out.

(The fact that the PMP text goes on to say "In particular, a portion of a misaligned store ...", is not limiting the above excerpt to just stores. It is just highlighting a notable particular example.)