satp/vsatp/hgatp configurations in multi-core system

[Qin0803]

in a multi-core system, some cores are configured as satp.mode=sv39 and others as satp.mode=sv48, does this scene exist? Is it possible to have the same ASID?

Is the same question consistent for hgatp? some cores are configured as hgatp.mode=sv39x4 and others as hgatp.mode=sv48x4, does this scene exist? Is it possible to have the same VMID?

[lfiolhais]

You are allowed to have multiple harts in a system with different supervisor
memory translation modes with the same underlying physical memory. The same
applies for hypervisor memory translation modes. It is the system software's
job to manage process migration between the different translation modes and
ensure they are mapped correctly; if that is desired.

ASIDs and VMIDs are hart-local. Their meaning within a system is
implementation-defined regardless of the translation modes used.

[Qin0803]

Q1: So when multiple harts in a system, different supervisor memory translation modes are allowed, which means that some harts may use Sv39, some harts may use Bare, and some harts may use Sv48 at the same time. Is that right?

Q2: another question, ASIDs and VMIDs are hart-local, So even if some harts use Sv39 and some harts use Sv48 at the same time, they may have the same ASID, is that right?

Q3: And even if some harts use Sv39x4 and some harts use Sv48x4 with different hypervisor memory translation modes at the same time, they may have the same VMID, is that right?

[lfiolhais]

Q1: So when multiple harts in a system, different supervisor memory
translation modes are allowed, which means that some harts may use Sv39, some
harts may use Bare, and some harts may use Sv48 at the same time. Is that
right?

Yes.

Q2: another question, ASIDs and VMIDs are hart-local, So even if some harts
use Sv39 and some harts use Sv48 at the same time, they may have the same
ASID, is that right?

Yes.

Q3: And even if some harts use Sv39x4 and some harts use Sv48x4 with
different hypervisor memory translation modes at the same time, they may have
the same VMID, is that right?

Yes.

[rock-ifly]

Hi Luís, Luís Fiolhais (@lfiolhais)
please help me to make these clear

Q1: So when multiple harts in a system, different supervisor memory
translation modes are allowed, which means that some harts may use Sv39, some
harts may use Bare, and some harts may use Sv48 at the same time. Is that
right?

Yes.

what is the running system?

1. are the answers right for the same HostOS running on a smt2 core(hart0+hart1)
   hart0 and hart1 are managed by the same OS, then they should have the same satp.mode. It seems impossible to config satp_hart0.mode=Sv39, and satp_hart1.mode=Sv48. It is unreasonalbe since the OS configureation should decide mode from the system poweron.
2. if hart0 run a VM（vmid0）， and hart1 run a VM(vmid1)， then the configuration of satp_hart0.mode=Sv39, and satp_hart1.mode=Sv48, seems OK.

Q2: another question, ASIDs and VMIDs are hart-local, So even if some harts
use Sv39 and some harts use Sv48 at the same time, they may have the same
ASID, is that right?

Yes.

Same as the above , it seems rediculous for the system without VM on. from the OS view, same asid should be the same process . only in the virtual machine, for different vmid, the same asid can work together.

Q3: And even if some harts use Sv39x4 and some harts use Sv48x4 with
different hypervisor memory translation modes at the same time, they may have
the same VMID, is that right?

Yes.

it is hard to understand the real application, they should not work in the same software system, if so, how does the software to distinguish the VM with the same vmid, and process with the same asid?

[aswaterman]

are the answers right for the same HostOS running on a smt2 core(hart0+hart1)

At this level, multithreading is a microarchitectural detail that should not be ISA-visible, so any answer that holds for a 2-core system with 1 hart per core also holds for a 1-core system with 2 harts per core.

it is hard to understand the real application, they should not work in the same software system, if so, how does the software to distinguish the VM with the same vmid, and process with the same asid?

It doesn't really matter what the application is; ASIDs and VMIDs are hart-private, so the ISA allows the same ASID/VMID pair to correspond to different translation modes on different harts.

[rock-ifly]

are the answers right for the same HostOS running on a smt2 core(hart0+hart1)

At this level, multithreading is a microarchitectural detail that should not be ISA-visible, so any answer that holds for a 2-core system with 1 hart per core also holds for a 1-core system with 2 harts per core.

it is hard to understand the real application, they should not work in the same software system, if so, how does the software to distinguish the VM with the same vmid, and process with the same asid?

It doesn't really matter what the application is; ASIDs and VMIDs are hart-private, so the ISA allows the same ASID/VMID pair to correspond to different translation modes on different harts.

Thanks for your reply. Andrew Waterman (@aswaterman)

So it is ISA's responsibility to allow the same ASID/VMID pair to correspond to different translation modes on different harts, but that how the system(hardware+software) use the asid/vmid is another thing,right?

e.g, on the view of the system, same VMID/ASID coulde be used to describe an unique VM, and also be used by different VM where a much bigger logic is used, like a supervisor domain（SDID>VMID>ASID).

[aswaterman]

Yes, that's right.

This ultimately results in a more scalable system in most cases. Since most processes only execute on a small subset of the harts, those processes only need to be allocated ASIDs on a small subset of the harts, and TLB shootdowns can be restricted to that subset, rather than being handled globally.

[lfiolhais]

Andrew Waterman (@aswaterman) thanks for stepping in and replying. Weekend and all...
I agree with the answers provided.

it is hard to understand the real application, they should not work in the same software system, if so, how does the software to distinguish the VM with the same vmid, and process with the same asid?

Consider a RISC-V system that allow different supervisor software that executes simultaneously (see the Smmtt extension proposal): one is executing hypervisor A, another is execution hypervisor B, and we are also executing an OS directly over M. In this instance, M-mode software will partition all upper privileges and each hart in the system can be programmed with completely different H and S states.

[rock-ifly]

Andrew Waterman (@aswaterman) Luís Fiolhais (@lfiolhais) Thanks for your reply.

[chara811]

Hi，Andrew Waterman (@aswaterman)
There is another scenario in a multi-hart environment. For instance, all these harts could mode= Sv39, but they each have their own page table mappings. Is that right?

[aswaterman]

Sure, in fact that’s the normal mode of operation.

[chara811]

Great, thanks a lot!