Allow offsets in TypedArray constructor which do NOT aline

[Fuzzyma]

The specs scpecify the behavior of offset when creating a new TypedArray with:

7. If offset modulo elementSize ≠ 0, throw a RangeError exception.

(https://tc39.github.io/ecma262/#sec-typedarray-buffer-byteoffset-length)

Imo this is a big limitation for memory efficient parsing of files.
Take for example a binary stl-file which holds a 3d model:

input.addEventListener('change', function (e) {
  const file = e.target.files[0];
  const reader = new FileReader()

  reader.onload = () => {
    const buffer = reader.result
    var offset = 0

    // First 80 Byte are header with description
    var description = String.fromCharCode.apply(null, new Uint16Array(buf, offset, 80/2))
    offset += 80

    // Next 32 Byte
    const numberOfTriangles = new Uint32Array(buf, offset, 1)
    offset += 4

    var normal = new Float32Array(buf, offset, 3)
    offset += 12

    //
    // reading vertices (all aligned)...
    // 
    
    // at some point you have attribute byte count
    // and you go
    offset += 2

    // BAAAM, no alignment anymore -> RangeError
    var anotherNormal = new Float32Array(buf, offset, 3)
  }

}

You could super easily wrap a Float32Array around every value you need and never touch the buffer or copy ANY data. However, at some point the Float32Array has to start at a byteOffset that is not aligned (see code above)

This throws the range error specified in the specs. But why do we have this limitation?
A TypedArray is only a view on the underlying data. It shouldnt matter at which byte I put my schablona to make sense of the underlying data. Now I need to copy over all data of the buffer which slows down stuff a lot when we are dealing with very big files.

• Why do we have this limitation?
• If there is no reason, can we allow offsets which are not aligned?
• If there is a reason, what is it and can we work around it?

[devsnek]

Why do we have this limitation?

Many CPU architectures do not allow or have severe performance penalties for unaligned accesses.

can we work around it?

We can! DataView can perform unaligned accesses. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView/getFloat32

[ljharb]

This seems answered; happy to reopen if not.

[Fuzzyma]

Thanks for the immediate Response. I am aware of DataView. But with a DataView, you end up copying every value into its own memory and thats explicitely NOT what I want. Your linked getFlot32 function is the best example:

// new memory allocation
var val = view.getFoat32(offset)

I am aware that CPUs indeed like aligned access. However, DataView does unaligned access, too and in that remark it is not different from a simple array access.

Beside that, the alignment also doesnt matter when I use an UInt8Array which spawns over multiple registers and set a random Byte. In case of a Float32Array you just would set 4 random Bytes instead (well technically they are next to eachother but you know what I mean).

I also can't imagine, that the perfomance penalties are worse, than copying literally every Byte of the buffer in its own variable - because thats what you do with DataView.
Reading data once into memory and working with that should bring a significant performance gain.

@ljharb Unfortunately my issue is not resolved

[devsnek]

dataview doesn't copy anything...

in that remark it is not different from a simple array access.

the difference is that allowing unaligned typed array views would introduce people to enormous performance penalties with no warning. on the other hand, I think most people expect that calling a function has some performance overhead at least.

[Fuzzyma]

no, not perse but there is a difference between this:

var normal = [
    view.getFloat32(offset),
    view.getFloat32(offset+4),
    view.getFloat32(offset+8)
]

and that:

var normal = new Float32Array(buf, offset, 3)

Can you see that in the first example I am allocating an array which is already present in a buffer when I just would put a Float32Array over the buffer?

the difference is that allowing unaligned typed array views would introduce people to enormous performance penalties with no warning. on the other hand, I think most people expect that calling a function has some performance overhead at least.

Drop a warning on the docs to use aligned offsets for best performance.
In some cases this performance drop is less than the overhead you get from copying everything

[bergus]

I also can't imagine that the performance penalties are worse than copying literally every Byte of the buffer in its own variable

The difference is when the penalty applies: when you copy the bytes into a new, aligned buffer, it costs once on construction, when you use getFloat32 on an unaligned buffer it costs on every single access.