Skip to content

args get validated but not function returns #71

@tobiasholler

Description

@tobiasholler

convex-test appears to validate function arguments, but not function return values against returns validators.
This causes tests to pass while real Convex runtime rejects the same function result.

Repro scenario

A query with a strict returns validator:

export const queryRoles = query({
  args: { search: v.optional(v.string()) },
  returns: v.array(
    v.object({
      _id: v.id("roles"),
      name: v.string(),
      permissions: v.array(permissionValidator),
    }),
  ),
  handler: async (ctx) => {
    // raw documents include _creationTime
    return await ctx.db.query("roles").collect();
  },
});

In real runtime, this fails return validation because _creationTime is extra.
In convex-test, this passes.

Source findings

In index.ts:

  • queryFromPath validates only exportArgs:
    • validateValidator(JSON.parse((func as any).exportArgs()), args ?? {});
    • around line ~1966
  • mutationFromPath and actionFromPath do the same for args only:
    • around ~1982 and ~1995
  • Query execution then goes through runQueryWithHandler (around ~1895), which wraps the function with:
    • queryGeneric({ handler: ... })
    • but does not apply func.exportReturns() from the original function definition.

Because the original function definition (with its returns) is not passed through, return validation is effectively skipped in tests.

Expected behavior

convex-test should enforce returns validators similarly to runtime, or clearly document that returns are not validated.

Impact

  • False positives in tests
  • Runtime-only failures
  • Contract drift between test and production behavior

Request

  • Validate returned values against exportReturns()

Environment

  • convex-test: 0.0.41
  • convex: 1.32.0
  • Bun + Vitest setup

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions