On these lines there is an assumption that bytes are pinned, which is absolutely does not have to be the case:

https://github.com/tweag/cardano-canonical-ledger/blob/177d9ed4b35f656a51200fd5e4a9518780fb0e10/mempack-scls/mempack-1.0/Data/MemPack/Extra.hs#L238-L240

I am not sure whether there exists an equivalent of hPutBuf for ByteArrays (probably not, since it is implemented in terms of system calls, which requires a pointer), so if there is not, then the safest thing to do would be to create a helper like this that checks whether ByteArray is pinned and make a pinned copy otherwise, which then in turn can be used as a Ptr