[Discussion] - faster validation

[overthemike]

I just discovered this lib - very cool - and I think I may have a way to make quick validation possibly even faster. I made a library a little while back for a different purpose that takes objects and hashes them based on structure and type (ignoring values). It might only work for certain common use-cases, but if performance is a big win, it might be something worth checking out. Here's a benchmark comparing it with JSON.stringify just for some context

The library is here: https://github.com/synpatico/genome (moved from https://github.com/overthemike/structure-id).

[joshmossas]

Hmm so if it's creating a hash of an existing object how does it handle values that can be null? Or something like a tagged union?

type ExampleType = {
  foo?: string;
  bar: number | null
};

type ExampleTaggedUnion = { type: "FOO"; data: string } | { type: "BAR"; data: number };

The hashing strategy is interesting but I fear it's probably too restrictive for Arri's use cases

[joshmossas]

I just ran a quick benchmark and it seems like Arri's compiled validators are faster compared to structure id (at least on my macbook)

Here's what the benchmark looks like. You can let me know if I'm using your library incorrectly.

const structureId = generateStructureId(goodInput);
void benny.suite(
    'Structure ID Test',
    benny.add('Arri', () => {
        assert(a.validate(ArriUser, goodInput));
    }),
    benny.add('Arri (Compiled)', () => {
        assert($$ArriUser.validate(goodInput));
    }),
    benny.add('StructureId', () => {
        assert(structureId === generateStructureId(goodInput));
    }),
    benny.cycle(),
    benny.complete(),
    benny.save({
        file: 'structure-id',
        folder: 'benchmark/dist',
        format: 'chart.html',
    }),
    benny.save({
        file: 'structure-id',
        folder: 'benchmark/dist',
        format: 'json',
    }),
);

[overthemike]

Hmm so if it's creating a hash of an existing object how does it handle values that can be null? Or something like a tagged union?

By itself, it wouldn't really. It would more serve as a fast 'yes' for objects whose value types won't change. I guess it could be updated to allow for empty type values, but I think that would go beyond the scope of what it's really for. I pictured it more like a fast 'yes' for arri when possible, but then let arri do what it does if not.

Here's what the benchmark looks like. You can let me know if I'm using your library incorrectly.

Yeah it's not quite correct, but I'll take the blame for that. I could have definitely explained it better.

benny.add('StructureId', () => {
    assert(structureId === generateStructureId(goodInput));
}),

This is unnecessary and redundant. The idea would be to run the function on the original object, then when new values come in, compare the hash to the hash of the object with the new values.

const origObj = {
  id: 1,
  name: 'something',
  sub: {
    foo: 'bar'
  }
}

const cachedStructure = generateStructureId(origObj)
// L0:17329588662-L1:4956270389-L2:6

const newObjValues = {
  id: 2,
  name: 'something else',
  sub: {
    foo: 'not bar'
  }
}

const newStructure = generateStructureId(newObjValues)
// Different values with same structure produces the same hash
// L0:17329588662-L1:4956270389-L2:6

if (newStructure === cachedStructure) {
  return true // fast check to see if new object is valid
}

// arri does validation if fast check doesn't work

So for your test, I would imagine it would be more like this:

const structureId = generateStructureId(origStructreObj);
void benny.suite(
    'Structure ID Test',
    benny.add('StructureId', () => {
        assert(structureId === generateStructureId(goodInput));
    }),
);

The idea I had behind it was to compare one hash string to another for a quick 'yes' validation instead of comparing multiple. If the hashing is slowing it down though, not a good fit. I'll take a more detailed look when I get a chance and if I find anything different, I'll let you know. Thanks for taking a look regardless!