xxHash functions not providing the correct output

[ryn9]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Problem

As originally reported here: vectordotdev/vector#23817 (comment) ...

I think both 64 bit outputs have a bug...
its looks like some type of truncation issue - where the last 2 digits are always 00 - when that should not be the case

using VRL playgroud....

code:

.foo_xxh32  = xxhash("foo","XXH32")
.foo_xxh32_128 = xxhash("foo","XXH3-128")
.foo_xxh32_64  = xxhash("foo","XXH3-64")
.foo_xxh64 = xxhash("foo","XXH64")

.test_xxh32  = xxhash("test","XXH32")
.test_xxh32_128 = xxhash("test","XXH3-128")
.test_xxh32_64  = xxhash("test","XXH3-64")
.test_xxh64 = xxhash("test","XXH64")

.xxxx_xxh32  = xxhash("xxxx","XXH32")
.xxxx_xxh32_128 = xxhash("xxxx","XXH3-128")
.xxxx_xxh32_64  = xxhash("xxxx","XXH3-64")
.xxxx_xxh64 = xxhash("xxxx","XXH64")

output:

{
	"foo_xxh32": 3792637401,
	"foo_xxh32_128": "161745101148472925293886522910304009610",
	"foo_xxh32_64": -6093828362558604000,
	"foo_xxh64": 3728699739546630700,
	"test_xxh32": 1042293711,
	"test_xxh32_128": "144184260470930790206475339950959648696",
	"test_xxh32_64": -7004795540881933000,
	"test_xxh64": 5754696928334414000,
	"xxxx_xxh32": 1978068271,
	"xxxx_xxh32_128": "50869038184720533382866996389792836481",
	"xxxx_xxh32_64": -4350627974171734000,
	"xxxx_xxh64": -7564316172907816000
}

Running this elsewhere ...

================================================================================
String: "foo"
================================================================================

┌─────────────┬──────────────────────────┬──────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Algorithm   │ Decimal                  │ Hexadecimal                          │ Binary                                                                                                                      │
├─────────────┼──────────────────────────┼──────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ XXH32       │ 3792637401               │ 0xe20f0dd9                           │ 11100010000011110000110111011001                                                                                    │
│ XXH64       │ 3728699739546630719      │ 0x33bf00a859c4ba3f                   │ 0011001110111111000000001010100001011001110001001011101000111111                                                    │
│ XXH3-64     │ 12352915711150947722     │ 0xab6e5f64077e7d8a                   │ 1010101101101110010111110110010000000111011111100111110110001010                                                    │
│ XXH3-128    │ 161745101148472925293886522910304009610 │ 0x79aef92e83454121ab6e5f64077e7d8a   │ 01111001101011101111100100101110100000110100010101000001001000011010101101101110010111110110010000000111011111100111110110001010 │
└─────────────┴──────────────────────────┴──────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

================================================================================
String: "test"
================================================================================

┌─────────────┬──────────────────────────┬──────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Algorithm   │ Decimal                  │ Hexadecimal                          │ Binary                                                                                                                      │
├─────────────┼──────────────────────────┼──────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ XXH32       │ 1042293711               │ 0x3e2023cf                           │ 00111110001000000010001111001111                                                                                    │
│ XXH64       │ 5754696928334414137      │ 0x4fdcca5ddb678139                   │ 0100111111011100110010100101110111011011011001111000000100111001                                                    │
│ XXH3-64     │ 11441948532827618368     │ 0x9ec9f7918d7dfc40                   │ 1001111011001001111101111001000110001101011111011111110001000000                                                    │
│ XXH3-128    │ 144184260470930790206475339950959648696 │ 0x6c78e0e3bd51d358d01e758642b85fb8   │ 01101100011110001110000011100011101111010101000111010011010110001101000000011110011101011000011001000010101110000101111110111000 │
└─────────────┴──────────────────────────┴──────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

================================================================================
String: "xxxx"
================================================================================

┌─────────────┬──────────────────────────┬──────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ Algorithm   │ Decimal                  │ Hexadecimal                          │ Binary                                                                                                                      │
├─────────────┼──────────────────────────┼──────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ XXH32       │ 1978068271               │ 0x75e6ed2f                           │ 01110101111001101110110100101111                                                                                    │
│ XXH64       │ 10882427900801735864     │ 0x97062684a9165cb8                   │ 1001011100000110001001101000010010101001000101100101110010111000                                                    │
│ XXH3-64     │ 14096116099537817804     │ 0xc39f76e8799dd0cc                   │ 1100001110011111011101101110100001111001100111011101000011001100                                                    │
│ XXH3-128    │ 50869038184720533382866996389792836481 │ 0x2645053829747c5af436909ed57deb81   │ 00100110010001010000010100111000001010010111010001111100010110101111010000110110100100001001111011010101011111011110101110000001 │
└─────────────┴──────────────────────────┴──────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Maybe you could add an option specify output format?
int (which would do the signed int, or fail for 128 bit)
string-decimal (which would return a string of the (unsigned) decimal representation)
string-hexadecimal (which would return a string of the (unsigned) hexadecimal representation)

VRL Program

VRL and/or Vector Version

N/A

Debug Output

Example

No response

Additional Context

No response

References

No response