Heap overflow in weight_packer issue

[GilesBathgate]

@ankan-ban There is a heap overflow in weight_packer. I have added code to highlight the problem area which prints a message. The output of ASAN without these changes is also similar to the following:

$ ./weight_packer config.json output model-7b.bin 1

Model params:- 
dim: 4096 
hidden_dim: 11008
n_heads: 32
n_kv_heads: 32
n_layers: 32
seq_len: 4096
vocab_size: 32000
rope_theta: 10000

Processing weights for layer: 0
Error: Heap overflow detected at index 352256
=================================================================
==28541==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f98d9dfb800 at pc 0x56475bbdf4e4 bp 0x7fff2f4caa00 sp 0x7fff2f4ca9f0
READ of size 4 at 0x7f98d9dfb800 thread T0
    #0 0x56475bbdf4e3 in repack_q_data(unsigned int*, unsigned int const*, int, int) (llama2_cu_awg/build/weight_packer+0x34e3)
    #1 0x56475bbdf6c0 in repack_q_weights(unsigned int*, unsigned int*, unsigned short*, unsigned int const*, unsigned int const*, unsigned short const*, int, int, int) (llama2_cu_awg/build/weight_packer+0x36c0)
    #2 0x56475bbdfbb9 in repackQWeightByName(_IO_FILE*, char*, char const*, unsigned long, unsigned long, bool) (llama2_cu_awg/build/weight_packer+0x3bb9)
    #3 0x56475bbe0a4d in main (llama2_cu_awg/build/weight_packer+0x4a4d)
    #4 0x7f98dfc29d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)
    #5 0x7f98dfc29e3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f)
    #6 0x56475bbde404 in _start (llama2_cu_awg/build/weight_packer+0x2404)

0x7f98d9dfb800 is located 0 bytes to the right of 1409024-byte region [0x7f98d9ca3800,0x7f98d9dfb800)
allocated by thread T0 here:
    #0 0x7f98e04b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x56475bbdeeed in repack_q_data(unsigned int*, unsigned int const*, int, int) (llama2_cu_awg/build/weight_packer+0x2eed)
    #2 0x56475bbdf6c0 in repack_q_weights(unsigned int*, unsigned int*, unsigned short*, unsigned int const*, unsigned int const*, unsigned short const*, int, int, int) (llama2_cu_awg/build/weight_packer+0x36c0)
    #3 0x56475bbdfbb9 in repackQWeightByName(_IO_FILE*, char*, char const*, unsigned long, unsigned long, bool) (llama2_cu_awg/build/weight_packer+0x3bb9)
    #4 0x56475bbe0a4d in main (llama2_cu_awg/build/weight_packer+0x4a4d)
    #5 0x7f98dfc29d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f)

SUMMARY: AddressSanitizer: heap-buffer-overflow (llama2_cu_awg/build/weight_packer+0x34e3) in repack_q_data(unsigned int*, unsigned int const*, int, int)
Shadow bytes around the buggy address:
  0x0ff39b3b76b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff39b3b76c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff39b3b76d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff39b3b76e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0ff39b3b76f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0ff39b3b7700:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff39b3b7710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff39b3b7720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff39b3b7730: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff39b3b7740: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0ff39b3b7750: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==28541==ABORTING