bug: missing field generates "missing-cell" errors additionnaly to the "missing-label" error

[pierrecamilleri]

Description

When providing data with missing fields/labels compared to the schema fields, and provided that the validation happens without "--schema-sync" (or soon, with default "fieldsMatch": "exact" property), a "missing-label" error is raised (expected), but also many "missing-cell" errors (unexected).

Expected behavior

There should not be a "missing-cell" error in the report, for two reasons :

• it does not add any useful information to the "missing-label" error
• it's use should be limited to irregular rows (e.g. missing value in a csv row), not for a missing field in the data.

To reproduce

a
1
2

{
  "resources": [
    {
      "name": "data",
      "path": "data.csv",
      "format": "csv",
      "schema": {
        "fields": [
          {"name": "a"},
          {"name": "b"}
          ]
      }
    }
  ]
}

frictionless validate dp.json

EDIT: same problem with extra-cell ! (and same origin)

[stefan1condeescu]

Hi! I’d like to work on this issue as my first contribution. I plan to reproduce the bug, add a regression test, and implement a small fix. Is the issue still available?

[pierrecamilleri]

Hi, thanks for your proposition. This issue is still available. I'd suggest to start from PR 1790 about fieldsMatch or wait til it is merged however to avoid conflicts.

[stefan1condeescu]

Thanks for the heads-up! I’ll wait for #1790 to be merged, then continue with this issue.

[pierrecamilleri]

#1790 has been merged.

A few hints on the current issue :

• in "test_validate_schema.py", tests need to be updated at least here
  and here.

• The bug probably occurs in table/header.py in the Header.get_expected_fields method, which deals with this problem when the
  header is matched by name. For extra fields it should create additional "any"-typed fields, for missing ones it should truncate the expected fields.