-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Description
convex-test appears to validate function arguments, but not function return values against returns validators.
This causes tests to pass while real Convex runtime rejects the same function result.
Repro scenario
A query with a strict returns validator:
export const queryRoles = query({
args: { search: v.optional(v.string()) },
returns: v.array(
v.object({
_id: v.id("roles"),
name: v.string(),
permissions: v.array(permissionValidator),
}),
),
handler: async (ctx) => {
// raw documents include _creationTime
return await ctx.db.query("roles").collect();
},
});In real runtime, this fails return validation because _creationTime is extra.
In convex-test, this passes.
Source findings
In index.ts:
queryFromPathvalidates onlyexportArgs:validateValidator(JSON.parse((func as any).exportArgs()), args ?? {});- around line ~1966
mutationFromPathandactionFromPathdo the same for args only:- around ~1982 and ~1995
- Query execution then goes through
runQueryWithHandler(around ~1895), which wraps the function with:queryGeneric({ handler: ... })- but does not apply
func.exportReturns()from the original function definition.
Because the original function definition (with its returns) is not passed through, return validation is effectively skipped in tests.
Expected behavior
convex-test should enforce returns validators similarly to runtime, or clearly document that returns are not validated.
Impact
- False positives in tests
- Runtime-only failures
- Contract drift between test and production behavior
Request
- Validate returned values against
exportReturns()
Environment
convex-test:0.0.41convex:1.32.0- Bun + Vitest setup
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels