RV32 has 34 bit physical address space and 32 bit long, so we cannot support all possible designs.
Since the physical addresses are usually aligned, we can shift right (2 bits or the alignment) to fit all possible values, or just use the two registers like some other SBI ecalls. I think it's also fine to drop support for highmem, but the intention should be explicitly documented.