Governed Structured Data (GSD)

This library assumes that structured data such as JSON, CSV, etc. are kept as TypeScript *.ts source files and then use this GSD library to generate their final formats.

The primary benefit is that no special JSON Schema or other schema management is necessary, all data structures are defined in TypeScript. By sticking with TypeScript, Visual Studio Code and other TypeScript-based editors can provide excellent syntax highlighting and error detection support. And, since TypeScript generates *.js it means that the final representations are usable on both the client and the server.

You can create either *.ts files that generate *.json files or, you can use existing *.json files with companion *.ts schema files that can validate any existing JSON files with any structure that TypeScript natively supports.

Usage - TypeScript that generates JSON

To use this library to generate valid, type-safe JSON, first create any TypeScript file and define any arbitrary data structures. Inside the TypeScript file, the content or any other variable can be generated organically or automatically with any level of complexity. Then, just add CliArgsEmitterat the bottom of the file.

import * as govnData from "https://denopkg.com/gov-suite/governed-structured-data/mod.ts";

export interface HomePage {
  hero: HeroContent[];
  sections: Section[];
  whyMedigyTitle: string;
  whyMedigyDescription: string;
  whyMedigyButton: string;
}

const content: HomePage = {
  hero: [...],
  sections: [...],
  whyMedigyTitle: someFunctionResult(...),
  whyMedigyDescription:  // multi-line templates are OK too
    `Medigy is crowd-sourced and peer network-based. Buyers 
     get access to rich content about the digital health products 
     they’re looking for. Influencers have a new place to build and
     engage with a community around their areas of expertise.`,
  whyMedigyButton: "Why Medigy?",
};

if (import.meta.main) {
  govnData.CLI(
    import.meta.url,
    govnData.defaultTypicalControllerOptions(content),
  );
}

Assuming the file is called my-data.ts then you can get help on the commands:

> deno-run my-data.ts --help

Governed Data Controller (GDC) ${$GDCTL_VERSION}. 

The GDC is designed to be "built into" each TypeScript file that is defined as "governed structured data" (a fancy
way of saying that data is strongly typed). The GDC can take an appropriately governed TypeScript and generate JSON 
plus do many other common data tasks such as validation and re-typing.

Usage:
  gdctl inspect
  gdctl json emit [<emit-dest>]
  gdctl json sync [--dry-run] [--overwrite]
  gdctl json type <json-src> --type-import=<url> --type=<symbol> [--dry-run] [--overwrite] [--instance=<symbol>] [--gsd-import=<url>] [--verbose]
  gdctl -h | --help
  gdctl --version

Options:
  <emit-dest>             The name of the file to emit, if it's just ".json" use same name as active file but force extension
  <json-src>              JSON single local file name or glob (like "*.json" or "**/*.json")
  --overwrite             If the file already exists, it's OK to replace it
  --type-import=<url>     The import where the primary TypeScript type definition is found
  --type=<symbol>         The TypeScript symbol that should be assigned the primary type
  --instance=<symbol>     The name of the TypeScript instance that should be assigned (default: "instance")
  --gsd-import=<url>      The import where the Governed Structured Data (GSD) library is found
  --dry-run               Don't perform any actions but be verbose on what might be done
  --verbose               Be explicit about what's going on (automatically turned on if using --dry-run)
  -h --help               Show this screen
  --version               Show version

For my-data.ts you can do the following to emit the typed data as JSON to STDOUT (if no filename is supplied, emit just sends the generated JSON to STDOUT):

deno-run my-data.ts json emit

It's even more powerful when the source data is at another location:

deno-run https://raw.githubusercontent.com/org/remote/master/test-data.gsd.ts json emit local-file.auto.json

Or, you can have it default it to a regular file with the same name at the same location if it's on the local file system (note that there's a space between the my-data.ts json emit and .json CLI argument):

deno-run my-data.ts json emit .json

The above will create my-data.auto.json to indicate that it's automatically generated JSON and shouldn't be edited directly.

Or, you can have it default it to a regular file with a different name:

deno-run my-data.ts json emit another-name.json

If the *.ts file was autogenerated by GSD using json type (see next section) then the following will show which JSON source generated the TypeScript file:

deno-run my-data.ts inspect

Usage - JSON that can use TypeScript to verify its schema

Sometimes the source JSON cannot be modified but we want to verify that JSON content matches a TypeScript Schema. The GSD library allows for what we call a "JSON Module".

Assume we want to verify that a JSON file matches the following interface, in a file called json-module.test-schema.ts:

export interface Expected {
  readonly text: string;
  readonly numeric: number;
}

Here's how we can easily validate it:

deno-run gdctl.ts json type untyped-data-typer.test-valid.json.golden --type-import="./untyped-data-typer.test-schema.ts" --type=Expected --gsd-import="./mod.ts" --validate && deno fmt

Note we're passing in --gsd-import=./mod.ts to show that we are using the local GSD library. If leave --gsd-import out, then the GitHub URL-based GSD module is used instead. If you're doing local testing or development then --gsd-import=./mod.ts is useful. Otherwise, leave it off to use the GitHub URL-based GSD module location.

If you ran the above code on JSON that looks like this, in a file called invalid.json:

{
  "text": "text value",
  "numeric": "bad number"
}

You would see this error:

TS2322 [ERROR]: Type 'string' is not assignable to type 'number'.
  numeric: 'bad number'
  ~~~~~~~
    at /invalid.ts:6:3

    The expected type comes from property 'numeric' which is declared here on type 'Expected'
      readonly numeric: number;
               ~~~~~~~
        at /json-module.test-schema.ts:3:12

The way the error is generated is that the GSD JSON Module library creates a dynamic TypeScript file, compiles it, and shows the resulting errors. Here's what the generated *.ts file looks like:

// Generated by GSD. DO NOT EDIT.
  
import * as govnData from "./mod.ts";
import type * as mod from "./untyped-data-typer.test-schema.ts";

// `instance` created on Sun Sep 27 2020 18:06:58 GMT-0400 (EDT) from:
//   untyped-data-typer.test-invalid.json.golden
//   Sat Sep 26 2020 18:05:47 GMT-0400 (EDT), 53 bytes

export const instance: mod.Expected = {
  text: 'text value',
  numeric: 'bad number'
};
    
export default instance;
    
export const retype: govnData.JsonRetyper = {
  provenance: { ... },
  jsonRetyperOptions: { ... },
}

if (import.meta.main) {     
  govnData.CLI(
    import.meta.url,
    govnData.defaultTypicalControllerOptions(instance, { retype: retype }),
  );
}

The JSON Modules feature allows you to use the full power of TypeScript to validate a JSON file against a verifiable structure instead of having to rely on less reliable strategies such as JSON Schema.

As part of the GSD generation process, a retype object with provenance is included so TypeScript and the GSD module has enough information to know where the content was sourced from.