Should we change the object schema?

[fabian-hiller]

Great news! Valibot v1 RC is just around the corner. There are only a few issues holding us back. One of them is issue #983, which may lead to a breaking change. That's why I want to discuss this with you before making a final decision. This post will explain the current situation and the reasons for the proposed changes.

The problem

Currently, Valibot does not distinguish between missing and undefined object entries. This leads to a mismatch between the input and output values of a schema and their types when TypeScript's exactOptionalPropertyTypes configuration is enabled.

import * as v from 'valibot';

// This throws no error and types `output` as `{ key?: string }`
const Schema = v.object({ key: v.optional(v.string()) });
const output = v.parse(Schema, { key: undefined });

// TypeScript thinks that `key` is a string if it is present
// but this is wrong because `key` is actually `undefined`
if ('key' in output) {
  const key = output.key;
}

The solution

The solution to this mismatch is to finalize PR #1013 and change the implementation of object and optional to distinguish between missing and undefined object entries. To allow missing entries, optional must be used as the outermost schema of an object entry. To explicitly allow undefined as a value, another schema function called undefinedable must be used.

const Schema = v.object({
  key1: v.string(),                               // key1: string
  key2: v.optional(v.string()),                   // key2?: string
  key3: v.undefinedable(v.string()),              // key3: string | undefined
  key4: v.optional(v.undefinedable(v.string())),  // key4?: string | undefined
});

The impact

As seen in the previous code example, defining an object entry that can be both missing and undefined gets a little verbose. This gets especially ugly if you define a default value for both cases.

const Schema = v.object({
  key: v.optional(
    v.undefinedable(v.string(), 'undefinedable_default'),
    'optional_default'
  ),
});

To fix this, we could add a new function that covers both. The problem is that there is a third schema function called nullable that also allows null values. So adding a new function to cover every possible combination of optional, undefinedable and nullable would result in 7 functions in total. I doubt this will make the API any better.

Let's discuss

What do you think? Should we distinguish between missing and undefined object entries and fully support TypeScript's exactOptionalPropertyTypes configuration? If so, should we remove nullish and only provide 3 strong primitives with optional, undefinedable and nullable? Or should we add 4 more functions to cover all possible combinations? If so, what names would you use?

Your opinion matters to me. I encourage everyone to share their thoughts. Even quick feedback like "I like this ... and I don't like that ..." is welcome and will help to shape Valibot's future API design. Please discuss with us on here on GitHub and share your thoughts.

[xcfox]

Would it be possible to add a new method v.exactOptional()? At the same time, we could make slight modifications to the implementations of v.optional() and v.nullish().

const Schema = v.object({
  key1: v.string(),                  // key1: string
  key2: v.optional(v.string()),      // key2?: string | undefined
  key3: v.nullish(v.string()),       // key3?: string | null | undefined
  key4: v.exactOptional(v.string()), // key4?: string
  key5: v.nullable(v.string()),      // key5: string | null
  key6: v.undefinedable(v.string())  // key6: string | undefined
});

In backend scenarios, I frequently need to serialize objects and store them in Redis or a database. As you can see, const foo = { bar: undefined } and const foo = { } are equivalent after JSON.stringify(foo). Due to this reason, I usually directly use foo.bar, foo.bar === undefined, or foo.bar == null to read values, while I almost never use delete foo.bar to remove values or "bar" in foo to check for their existence.

I believe that there are only a few cases where it's necessary to distinguish between const foo = { bar: undefined } and const foo = { }. In such situations, the project should configure exactOptionalPropertyTypes and use v.exactOptional() instead of v.optional().

[bachmacintosh]

Coming from someone who does use exactOptionalPropertyTypes I think something like this approach would be alright. Even before I used that config, most of the time I wouldn't notice a present undefined property because I always used typeof foo === "undefined" to narrow it out, but the few times it does matter (Object.keys comes to mind), it wouldn't be much of a hassle to use a different schema.

I'm on the fence about it being named strictOptional instead to match the other strict schemas (and make it easy to find in the docs), though exactOptional does match up with the TSConfig option.

[Bilboramix]

I like a lot this proposal as it would permit to void breaking change.
I also rely a lot on serialization so this problem has not been there for me.

[fabian-hiller]

Thanks for your idea! I really like that it mainly keeps things as they are, and only offers an additional option with exactOptional for people who care about that detail.

[sandros94]

I also join this, as I also serialize almost everything.
And I also agree with @EltonLobo07's reply below about the fact that I do tend to use different defaults based on it being null or undefined (v.optional(v.nullable(v.string(), 'null_default'), 'undefined_default')), so I already see it being useful to have.

Voting for:

• keep optional, undefinedable and nullish as they are
• add strictOptional, focusing on API naming (those that know about exactOptional already understand the difference)

Yes, strictOptional + nullable could be verbose, but I don't see how much it will be used

[ivands]

I agree with @xcfox.
Changing optional to only mean the prop is missing wouldn't be that useful.

We are using Valibot to do input validation of our API's.
As the API consumer you would be super confused if passing undefined as object property would error but passing no property would pass.
This could also be super annoying when making an API client because the following code would always fail:

const createPost = async (title: string, message?: string) => {
  // message will always have an undefined value instead of a missing message prop.
  await client.createPost({ title, message })
}

You could also just strip away undefined properties from validated object XD.
That way TS would be correct.

[genki]

v.undefinedable(v.string()) and v.union([v.undefined_(), v.string()]) is different?

[fabian-hiller]

The validation logic and types are similar but undefinedable allows you to pass a default value that overwrites the input in case it is undefined.

[genki]

Thx, I understand.

[EltonLobo07]

Since there are many cases where optional is closely related to null and undefined values, I think it is best to include schemas related to null and undefined in optional (check the examples at the end of the message). There are some benefits associated with this design:

1. Even though there is still some amount of verbosity, the schema is easy to read. There are no hidden rules involved. Example: understanding the difference between optional and strictOptional (the 1st comment of this GitHub discussion).
2. There won't be a need to create additional schemas to solve the common default value problem.

// NOTE: All of the types mentioned are output types

// `optional` within `object`
const ObjectSchema = v.object({
  // without default value
  k1: v.optional(v.string()),                         // k1?: string
  k2: v.optional(v.string(), "none"),                 // k2?: string
  k3: v.optional(v.string(), "undefined"),            // k3?: string | undefined
  k4: v.optional(v.string(), "null"),                 // k4?: string | null
  k5: v.optional(v.string(), "nullish"),              // k5?: string | undefined | null
  // with same default value
  k6: v.optional(v.string(), "none", "default"),      // k6: string
  k7: v.optional(v.string(), "undefined", "default"), // k7: string
  k8: v.optional(v.string(), "null", "default"),      // k8: string
  k9: v.optional(v.string(), "nullish", "default"),   // k9: string
  // with different default values (very rare but possible)
  // for all the cases, the type of the key-value pair will be - key: string
  k10: v.optional(
    v.undefinedable(v.string(), "undefined_default"), 
    "optional_default"
  ),                                                 
  k11: v.optional(
    v.nullable(v.string(), "null_default"),
    "optional_default"
  ),                               
  k12: v.optional(
    v.nullish(v.string(), "nullish_default"),
    "optional_default"
  )                  
});

// `optional` outside `object`
// `optional` in schemas 1, 2 and 3 acts like an identity function (won't change the output)
const Schema1 = v.optional(v.string());                         // string
const Schema2 = v.optional(v.string(), "none");                 // string
const Schema3 = v.optional(v.string(), "none", "default");      // string
const Schema4 = v.optional(v.string(), "undefined");            // string | undefined
const Schema5 = v.optional(v.string(), "undefined", "default"); // string
const Schema6 = v.optional(v.string(), "null");                 // string | null
const Schema7 = v.optional(v.string(), "null", "default");      // string
const Schema8 = v.optional(v.string(), "nullish");              // string | undefined | null 
const Schema9 = v.optional(v.string(), "nullish", "default");   // string 
// using `optional` like shown below does not make sense as optional's default is not used at all
// so this and related cases are naturally avoided
const Schema10 = v.optional(
  v.undefinedable(v.string(), "undefined_default"),
  "optional_default"
);

[fabian-hiller]

Thanks for the idea Elton! I hadn't thought about an API design like that at all. That's why I love these discussions and brainstorming with the community.

[typed-sigterm]

Personally I think v.optional should make { key: string } into { key?: string | undefined }, just like Partial.

[fabian-hiller]

Not if exactOptionalPropertyTypes is enabled: https://www.typescriptlang.org/play/?exactOptionalPropertyTypes=true#code/JYOwLgpgTgZghgYwgAgCrIN4FgBQzkDWEAngFzIDOYUoA5gNy4C+uuYxADiqgEzIC8yAApwoYYHAA2AHlQA+ekA

[Hugos68]

Just wanted to say I really like the proposed solution and like correctness much more over n less characters to type.

[fabian-hiller]

Hey everyone, thanks for your contributions! I followed @xcfox's idea in PR #1013. You can install it with the command below and report back if you like. A new official version on npm will follow soon.

npm i https://pkg.pr.new/valibot@1013

[fabian-hiller]

Valibot v1.0.0-beta.13 is available