Skip to content

Make scamper1 mda output to single JSON object (autoloadable) #169

@stephen-soltesz

Description

@stephen-soltesz

The scamper1 traceroute format by default is four JSON objects, creating a JSONL file per trace. This format is not supported by the autoloading system. Ideally, traceroute-caller would support a native JSON output format for scamper1 traces.

The native scamper structure uses a double array, [][]ScamperLink which the bigquery.InferSchema cannot work with. The etl parser transforms the raw object into a new object that is structured compatibly with BigQuery schemas.

To achive an autoloadable, JSON output for scamper mda traces, this transformation process must be local to the traceroute caller. This does not necessarily imply we must remain backward compatible with scamper1 schema, only that the transform is local and the output is autoloadable JSON.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions