Apl 215 implement error messages for x justiz data validation

[m0dh4x]

I wired the xJustiz converter's datatype C into one page schema field (sachverhaltBegruendung) as a first draft :)

the wrapper is applied per field, so flows that don't need it stay untouched. text lives in translations.ts

[weilbith]

A2J Platform x XJustiz-Converter 🥳

Thanks for starting to play around with it to get a first feeling. Super valuable feedback and a first little bug we gonna fix. 🙏

We already had a quick call and discussed the basics here. I also took the discussion back into the Court Communication team. Out goal is react as early as possible to have the integration as seamless as possible between both components. Communication as a skill. 👷 😉

The two major points still remain. Check out the respective comments I left. I tried too be quite detailed here. Let me know if it works for you. Directly pairing together is always welcome. Though, for this one I had to do some research and experiments, which I consider as part of the service from the Court Communication team.

[weilbith]

Interesting logic here. I think this can be improved a little to make the integration feel more "seamless".

Theoretically, the datatypes can be used directly in the page schemas, because Zod supports the Standard Schema out of the box. To apply customization on validation issue messages, the probably best approach would be something like this:

import {
  datatypeA as originalDatatypeA,
  datatypeB as originalDatatypeB,
  // ...
} from "@digitalservicebund/a2j-xjustiz-converter/nachricht/zahlungsklage";

export const datatypeA = originalDatatypeA.customize({ /* ... */ });
export const datatypeB = originalDatatypeB.customize({ /* ... */ });

Anywhere in the application you can now import the datatypeA from here and have it plug and play. I suspect that IDEs will even prioritize this definition on auto import. You might even wanna use the no-restricted-imports Oxlint rule to prevent this by accident.

However, you need some extension. Because you need to combine the schemas from the library with custom schemas. Like your example of a required string with a maximum length and datatypeA. Theoretically, this should be possible, using just normal Zod operators to combine multiple schemas. So you could do it like:

import { datatypeA } from "~/services/validation/xjustiz/xjustizDatatype";

export const geldEinklagenKlageErstellenPages = {
  somePage: {
    sachverhaltBegruendung: {
      pageSchema: {
        // Here it comes:
        sachverhaltBegruendung: stringRequiredMaxSchema({}).pipe(datatypeA),
      },
    },
  }
}

Notice to use the pipe operator and NOT plain and. The latter dismisses the type transformation that is essential for these datatypes. Using the and operator would result into a plain string value as parse result. Using the pipe operator make the final parse result preserve the transformed type of DatatypeA that will be essential later to compose an XJustiz-Message. Else this here kinda void and nothing.

Warning

Update: Release it out

Please notice, that there is actually a bug in the library right now. The (transformed) output type is not correctly inferred via the Standard Schema. We gonna release a patch for this as soon as possible.

Unfortunately, Zod does not support Standard Schemas to be used with its operators. So the above code does actually not work like this straight away. I looked into the Zod code base and we only way to make it work properly, but "converting" Standard Schemas to Zod schemas. The following code snippet should do the job and keeps this contained in the already established module for the customization:

import {
  datatypeA as originalDatatypeA,
  datatypeB as originalDatatypeB,
  // ...
} from "@digitalservicebund/a2j-xjustiz-converter/nachricht/zahlungsklage";
import { z } from "zod";

export const datatypeA = convertStandardSchemaToZod(
  originalDatatypeA.customize({ /* ... */ }),
);

export const datatypeB = convertStandardSchemaToZod(
  originalDatatypeB.customize({ /* ... */ }),
);

/**
 * While Zod supports Standard Schemas natively, it doesn't allow them to be used
 * with operators like `and` or `pipe`. For example,
 * `someZodSchema.pipe(someStandardSchema)` is not allowed. This function takes
 * a Standard Schema and constructs a fully integrated Zod schema from it.
 * Doing so, it takes into account possible input to output transformations by
 * the validation function of the Standard Schema.
 */
function convertStandardSchemaToZod<Input, Output>(
  schema: StandardSchemaV1<Input, Output>,
): z.ZodType<Output, Input> {
  return z.any().transform((input, context) => {
    const result = schema["~standard"].validate(input);

    if (result instanceof Promise)
      throw new Error("Asynchronous schemas are not supported");

    if (result.issues) {
      result.issues?.forEach((issue) => context.addIssue(issue.message));
      return z.NEVER;
    } else {
      return result.value;
    }
  });
}

Notice, that I depend on the StandardSchemaV1 type here. You need to decide how to get it into your code base. The most direct version is to add the @standard-schema/spec as a dependency. Zod only exposes the $ZodStandardSchema type, which doesn't work here. Alternatively, the Standard Schema suggests that you can also copy relevant parts directly into the code base, at is it quite minimal. If you should feel uncomfortable with any of these solutions, we can also export the type from the XJustiz-Converter.

[m0dh4x]

Thanks @weilbith, this is really helpful — I'll go with convertStandardSchemaToZod.

I will try it against the current version and check if it works as you describe: stringRequiredMaxSchema({...}).pipe(datatypeC) keeps trim, required and max intact and produces our customized message.

On the dependency thing. I think it's enough to put it to the devDependencies cause: @standard-schema/spec is already in our tree transitively.

[weilbith]

I'd like to push back on this in the most polite way. Let me explain why.

To be sure about it, I checked the Zod code base again (what a messy open source project). I also took a (rather quick) look into rvf for this. The reality seems to be, that validation errors are no meant to carry additional metadata. I know this seems frustrating (at least I get frustrated from it), but from all the libraries I know, the ecosystems seems to have decided on that input validation errors are plain strings.
While Zod has at least some theoretical support with the $ZodCustomError type with the params property of a plain Record, this information doesn't carry down the forms. There is also no good way to access the params in a secure way.

In result, you are fighting against the current idiomatic patterns. That means it costs effort.
This is a strong indicator for me to communicate this back into the team. Make the complexity clear. Ask the designers, how much impact this has on the user experience in contrast to a nicely formatted string. Do you have the user research (experience) to know if is worth the effort and complexity to maintain it, having structured error messages.
If the answer remains yes, make this visible to product management too. To enable this design, it requires to establish a proper mechanism to do so. The frameworks/libraries don't provide a solution to this. So you need a custom one. And that should probably be an explicit solution that is expressive with clean code, that is tested, documented, etc. I'd personally advise against an ad-hoc solution that just works right now. It likely contributes to the technical debt and increases the maintenance effort long term. If you need to go down this road, I'd suggest to do it iteratively. First go with the plain text message to provide user value quickly (unless design insists not to). Then have two tickets in your backlog. First adjust the code base with the capability to have structured validation errors in general. Make it a proper concept and feature of your product. There are some interesting solution approaches to achieve this. Finally, iterate the input validation to produce and display structured validation errors to the user.

In case I'm completely off here, priorities are different, and you simply have to do it right now, I'd go at least with an "in your face" solution here. Maybe something which looks like the following snippets. This isn't the best code in the world and can definitely be improved. It's just about the general idea.

xjustizDatatype.ts:

/** Document what the hell is going on here... */
function encodeValidationErrorAsJson(data: unknown): string {
  return JSON.stringify(data);
}

/** Oh hell document what this escape mechanism is all about... */
export function tryDecodeValidationErrorFromJson<ExpectedOutput>(error: string): ExpectedOutput | undefined {
  try {
    return JSON.parse(error) as ExpectedOutput;
  } catch {
    return undefined; // Yikes... 🤕
  }
}

export datatypeA = originalDatatypeA.customize({ invalidCharacters: encodeValidationErrorAsJson });

InputError.tsx:

export const inputErrorMessage = (
  error: string | null,
  errorMessages?: ErrorMessageProps[],
) => {
  if (!error) return null;

  const parsedValidationError = tryDecodeValidationErrorFromJson</* What actually ..? */>(error);
  const isJsonEncodedError = parsedValidationError !== undefined;

  if (isJsonEncodedError)
    // ...
  );

  return errorMessages?.find((err) => err.code === error)?.text ?? error;
};

[m0dh4x]

I already tried the plain-string version — I've had the same issue with the list design. It was one line in the customize callback, putting the final German sentence plus the characters straight into the message: no decoding, no component changes, because the existing ?? field.error() fallback renders it as-is. It's whether the characters need to be individual list items, or whether an inline enumeration does the job. I'll take that back to service team with the trade-off — one line versus four files.

To be honest, the only minor issue I have with this solution is that it can be a bit hard to read in a few cases. See image below. But I think that’s really a rare case.

[m0dh4x]

Something I noticed while digging into this together with @weilbith

The converter works with branded types — like DatatypeC instead of string — so the message generator only accepts values that actually went through a factory. That brand doesn't survive our session, though: we serialize the form data on every step, and unpack hands us back a Record<string, unknown>. The type doesn't travel with the data — at unpack the compiler is blind, it only knows what we told it.

So by the time we build the xJustiz message we're holding plain strings, and the generator won't take them. We could assert our way past that with as DatatypeC and nothing would break — which is exactly the problem, because it could fail invisibly: unvalidated data could flow into the message and we'd only notice downstream, or not at all until it reaches the court. The compile error is the safeguard here.

So my take is that we'll need to call the factories again when composing the message — that's the only way to get branded values in that process.

[m0dh4x]

I'm going with the branded variant in this PR, but the alternative is worth recording, because the argument for it isn't weak.

Since a branded value can't survive our session (see my other comment), the factories have to run again where the message is composed — in both variants. Which raises the question of what carrying the brand through the form layer buys us, if the guarantee has to be re-establish ed anyway.

The alternative: apply the datatype as a check

At the call site it's one word:

// this PR
stringRequiredMaxSchema({ max: TEXTAREA_MAX_LENGTH }).pipe(datatypeC)

// alternative
stringRequiredMaxSchema({ max: TEXTAREA_MAX_LENGTH }).check(datatypeC)

The module behind it

type Parse = (value: string) => { issues?: readonly { message: string }[] };

/** Turns a refined type factory into a Zod check on a string schema. */
const characterCheck =
  (parse: Parse): z.core.CheckFn<string> =>
  (ctx) => {
    const result = parse(ctx.value);
    if (result.issues)
      ctx.issues.push({
        code: "custom",
        input: ctx.value,
        message: result.issues[0].message,
      });
  };

export const datatypeA = characterCheck(
  originalDatatypeA.customize({ invalidCharacters }),
);
export const datatypeC = characterCheck(
  originalDatatypeC.customize({ invalidCharacters }),
);
// ... B, D, E

Same five named exports, same customization, same error message. The value stays a string, so convertStandardSchemaToZod and the @standard-schema/spec dependency both disappear — that machinery exists purely to carry the branded output type through Zod. The fixtures stay untouched as well.

The brand would then be established where the message is composed, using the factory from the library directly:

What stays the same either way

• The factories run again when composing the message — not optional, since the brand doesn't survive serialization into the session.
• The compiler still catches a field we forgot to parse there, or parsed with the wrong datatype. That protection comes from the generator's signature (zahlungsklage(name: DatatypeA, …)), not from the page schema.
• afaics: Invalid user input can't reach the boundary in either case

Open question for the team

The trade is: fixtures that go through the factory are verified test data — the compiler rejects values that couldn't occur in production — at the cost of ~154 assignments across 16 files. Plain literals cost nothing and assert nothing.

I've gone with the branded variant here so there's something concrete to look at, but I'd rather we decide this together before it goes beyond the single field it covers right now.

[weilbith]

Quick heads up:
The already announced renaming is done now. The old @digitalservicebund/a2j-xjustiz-converter package got deprecated and from version 0.6.0 on it is named @digitalservicebund/a2j-xjustiz-bridge.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
60.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud