`expectType` "identical" description misleading for TS type usage, excess properties in arg not reported

[karlhorky]

Hi, first of all, thanks for tsd, looks useful!

Looking at the Assertions -> expectType section in the readme, it appears that expectType<T>(expression: T) should assert that the types are identical:

expectType<T>(expression: T)

Asserts that the type of expression is identical to type T.

Scoping only to the context of the tsd TypeScript types (editor integration, tsc checks), "identical" is misleading: the type equality is only in a single direction, not bidirectional.

For example, the argument below with the excess property x is not reported as a TypeScript error (contrasted against expect-type, which correctly errors):

import { expectType } from "tsd";
import { expectTypeOf } from "expect-type";

type User = { id: number; name: string; email?: string; };
type UserExcess = { id: number; name: string; email?: string; y: string; };

type User2 = { id: number; name: string; email?: string; };
type UserExcess2 = { id: number; name: string; email?: string; x: string; };

expectType<User>({} as unknown as User2); // Correct: No error reported
expectType<UserExcess>({} as unknown as User2); // Correct: Error reported
expectType<User>({} as unknown as UserExcess2); // 💥 Incorrect: No error reported (false negative)

// expect-type `expectTypeOf().toEqualTypeOf()` gets this right
expectTypeOf<User2>().toEqualTypeOf<User>(); // Correct: No error reported
expectTypeOf<User2>().toEqualTypeOf<UserExcess>(); // Correct: Error reported
expectTypeOf<UserExcess2>().toEqualTypeOf<User>(); // Correct: Error reported

TypeScript Playground

Suggested Solution

1. Change the "identical" in the documentation to "assignable" or "compatible" or similar to avoid the word "identical" (or describe that it's only single-direction with other words)
2. Possibly also: I saw that the implementation of expectType is the same as the implementation of expectAssignable - should one of those two be dropped?
3. Possibly also: Offer a true "identical" utility (maybe named expectTypeStrict?) which will check in both directions, like expectTypeOf<>().toEqualTypeOf<>() from expect-type

[karlhorky]

cc @tommy-mitchell in case this fits into the v1.0.0 issue

• Towards v1.0.0 #196

[mrazauskas]

Sounds similar to #200.

If I get it right, you complain about typings, not about the behaviour of tsd.

Might sound bold, but the typings of expectType() and other assertion functions are incorrect per se. These are no-op functions. Thinking from compiler (or language service) perspective these functions are incorrectly typed. So there is one more solution: remove those typings all together.

I do understand why these generic types are attractive for the users and what makes expect-type popular. Indeed, immediately visible red squiggles are nice to have, but the problem is that error messages are cryptic or even misleading. Well.. because typings of those generic functions are not correct from compiler perspective. I doubt that can be fixed. The compiler is right.

(Obviously expect-type is popular, but why users do ignore the fact that error messages are unreadable, is a mystery for me. Correctness and quality is not important in testing? Probably I’m too conservative.)

What’s is the solution? I think the interesting field to explore is static analysis from type test runner perspective (instead of compiler perspective). tsd made a good start in this direction. That is rather involving job, but a lot can be done by analysing text (similar how the compiler does that). For instance, it is possible to check messages suppressed by // @ts-expect-error. Not only "there is some error", but "this is the exact error". Of course, this is what I find interesting to play with. Just an opinion.

[karlhorky]

Hmm, although I find multiple things you wrote above quite interesting, not sure I'm understanding the part that is responding to my issue above.

My main point is the word "identical" seems misleading to describe successful comparison of two object types, one of which has an excess property vs the other one.

From my issue description above:

type User = { id: number; name: string; email?: string; };

type UserExcess2 = { id: number; name: string; email?: string; x: string; };

expectType<User>({} as unknown as UserExcess2); // 💥 Incorrect: No error reported (false negative)

expect-type allows this to be reported as a problem, admittedly with an error which could be improved (the actual error message is the Expected: never, Actual: string in the last line):

Type 'User' does not satisfy the constraint '{ id: number; name: string; email: string | undefined; x: "Expected: never, Actual: string"; }'.
  Property 'x' is missing in type 'User' but required in type '{ id: number; name: string; email: string | undefined; x: "Expected: never, Actual: string"; }'. (2344)

I think the library that allows for bidirectional checking when being marketed as "identical" is the more correct one, from a superficial user level. This is where the scope of my issue above stops actually. Just to A) make tsd's docs not misleading B) possibly make tsd's implementations correct and bidirectionally strict.

I suspect that you know all of this already, and that you're digging in to the more philosophical correctness of both of the implementations. Not to detract from that (also really valuable!), but not the scope of my issue reported above.

[mrazauskas]

My main point is the word "identical" seems misleading to describe successful comparison of two object types

Is it when you only look at the editor or when you run tsd?

[karlhorky]

Oh, indeed I meant on TypeScript type level - editor integration, tsc checks, etc.

Maybe my issue is incorrectly worded to dismiss the CLI part of tsd.

[mrazauskas]

Yep. What I try to say: the tsds static analysis part is correct, only types are misleading. Remove them and all is solved. The rest of my comment (the motivation) was probably too verbose. Sorry about that.

By the way, tsds documentation does not say you can use tsc with its assertions. So, not a defect.

[karlhorky]

I reworded the issue title and description to be clear that this is about the TS types.

What I try to say: the tsds static analysis part is correct, only types are misleading.

Understood! That makes sense to me, although I haven't deeply verified it.

Remove them and all is solved.

My suggestion here would be the other direction - adopt types similar to expect-type (maybe with improved error messaging) to allow the checks in TypeScript type-level too.

By the way, tsds documentation does not say you can use tsc with its assertions. So, not a defect.

Ok, maybe it's a feature request then. I suppose it's my perception that this would be an expected part of tsd's interface, especially since the types do work for other situations.

[mrazauskas]

adopt types similar to expect-type

Why not to use expect-type instead?

---

What I try to say is that the strong side of tsd is static analysis. Take a look, expect-types list of limitation. Only 1. and 4. are reproduce with tsd and these cases work correctly with tsd (via CLI).

There are other issues not included in the list: playground, playground, playground. Only the last playground is reproducible with tsd (note that exactOptionalPropertyTypes: true must be set) and it works correctly with tsd (via CLI).

I say that the other cases are not reproducible, because tsd does not have such matchers. If those would get implemented one day, those would also work as expected. (Not a speculation, because an alternative test runner has working implementation. More details are out of scope here, I guess.)

The static analysis does 100% better job than generics. (Actually, tsd fails to do a good job because of expectType() generics, see: #141).

Also note expectNotType() has no editor feedback via generics. I cannot explain why the implementation is different, but I think this is correct one.

---

Of course it is the maintainers decision, not mine. Making noise again, sorry. And thanks for interesting discussion.

[karlhorky]

Yeah, thanks for the discussion! Cool to see the details behind expect-type and generics limitations 👍 I can agree that they are not perfect.

However, that doesn't mean that adding them would detract from the overall quality of tsd. If the user-facing generics / TS types cause problems for the tsd CLI's static analysis, one idea would be splitting them (eg. into simple library-facing types and more correct user-facing types).

Why not to use expect-type instead?

I think expect-type would probably be the tool of choice currently for TS type-level static analysis on "identical" types.

But regardless, I'm reporting this to tsd because I think tsd has the opportunity to improve. I think that TS types are part of the public API / interface for libraries and in the case of tsd, users will expect they work in the same as the details in the readme, even if the types / TypeScript support are not explicitly documented.

Also, to return to your argument about the complexity of the error message reporting and limitations of the types in expect-type, maybe tsd's version could be better.

[mrazauskas]

one idea would be splitting them (eg. into simple library-facing types and more correct user-facing types)

That could work. They can change resolution of the typings file while creating program.

I think tsd has the opportunity to improve

Totally agree. tsd needs a lot of improvement. (As you already mentioned: #196.)