How to document Key-value pairs in Data Package

[mirkov]

Hello, first post here :-)

I would like to package data files generated by diagnostic tools in a semiconductor fab.

The tools generate both tabular data (in CSV format), and meta-data files in a text format as key-value pairs

Obfuscated Example:

Algorithm version, V0.3
Wafer ID, sdo845743
Operator, Joe Smith

I went over the Data Package documentation and have not found ways to document this type of content.

Do you have any suggestions?

Thank you,

[roll]

Hi, I think it might be a resource like this:

{
  "path": "table.csv",
  "dialect": {
    "header": false,
    "skipInitialSpace": true
  },
  "schema": {
    "fields": [
      {
        "name": "key",
        "type": "string"
      },
      {
        "name": "value",
        "type": "string"
      }
    ]
  }
}

[roll]

$ dpkit table explore -p .user/datapackage-1131/package.json -r table
┌  Explore table
│
◇  Loading package
│
◇  Loading table
│
◇  Loading schema
│
◇  Normalizing table
╭────────────────────────────╮
│     key          value     │
│   (string)      (string)   │
│  Algorithm        V0.3     │
│  version                   │
│   Wafer ID      sdo845743  │
│                            │
│   Operator      Joe Smith  │
│                            │
╰────────────────────────────╯

[mirkov]

Thank you for the reply.

Initially my thought was also to specify it as a two-column table with a comma "," as a separator.

But with that, I cannot capture meta-data that is associated with each row. In this instance:

• For the algorithm, I could specify documentation on the versioning convention
• The Wafer ID also has a convention that includes a employee number and week of the year

I derived a "dictionary" specification for which copied a lot from the table specification. I can post that if there is interest.

Thanks again for your attention,

[roll]

It is possible to transpose this table and describe it with standard Table Schema? Or maybe just storing it as a JSON document with JSON Schema for context

[mirkov]

I write this as I travel, so I cannot reply in detail.

First, I cannot transpose the table. but, I can specify the table format as transposed. So I need to introduce a new field in the table format like: transposed=true.

Everything else stays the same, except for this new part of the table format specification.

I will clean up my thoughts when I get back