Please explicitly list "valid" characters

[yarikoptic]

ATM https://hed-specification.readthedocs.io/en/latest/Appendix_B.html#character-invalid says

a. An invalid character (character code < 32 or 127 <= character code < 160) appears in a HED string.

Humans rarely have ASCII table embedded in their brains, and require computing assistance. It would be great to spell out here what are those valid ones, so as "Valid characters are alpha-numerics of arbitrary casing, and ..."

E.g. in my case I got

        [ERROR] HED_ERROR The validation on this HED string returned an error.
                VALUE_INVALID
                /sub-0132/ses-04/func/sub-0132_ses-04_task-fractional_acq-mb8_run-01_events.tsv - ERROR: [VALUE_INVALID] Invalid placeholder value for tag "Organizational-property/Control-variable/reaching?". TSV line: "2". (For more information on this HED error, see https://hed-specification.readthedocs.io/en/latest/Appendix_B.html#value-invalid.)
                /sub-0132/ses-04/func/sub-0132_ses-04_task-fractional_acq-mb8_run-01_events.tsv - ERROR: [VALUE_INVALID] Invalid placeholder value for tag "Organizational-property/Control-variable/reaching?". TSV line: "3". (For more information on this HED error, see https://hed-specification.readthedocs.io/en/latest/Appendix_B.html#value-invalid.)

                13822 more files with the same issue

        Please visit https://neurostars.org/search?q=HED_ERROR for existing conversations about this issue.

so I assume it is about "?"? but that one seem to be code 64, which seems to be not an "invalid", thus a valid one?

Listing valid ones explicitly would help to get me an answer

[VisLab]

This should have had a message with the character that was invalid and why. Some tags that take # have value classes associated with them (e.g.., Label is of name class -- if you want UTF-8 you have to use Parameter-value). This didn't used to be enforced, but now we think we have complete validation. We am starting to go through all of the places were errors are detected to make sure that all the needed information is brought forward in the error messages. Could you please send me a sample of the tsv file and the JSON sidecar that went with it so we can track this down?

@yarikoptic Also, we have almost completed a browser-based HED validator based on the HED Javascript validator. It is available as a preview at https://www.hedtags.org/hed-javascript/.

[yarikoptic]

I had difficulty pushing to OpenNeuro, so pushed for now to https://github.com/spatialtopology/ds005256 .Let me know if any data file you need is not available.

[VisLab]

@yarikoptic The error that you were having (as you figured out) was because a blank was used in a column value that was substituted for # in Label/# and Label is of name class. I have made a complete pass over all the error messages in the JavaScript validator, and now the messages are more specific and hopefully much more informative. I started working on improving the information in the specification, but that is going to take longer.

The changes are available in the browser-based https://www.hedtags.org/hed-javascript/, but won't get into the BIDS validator until the next release (hopefully a PR will be coming in a couple of weeks). We'd really appreciate your trying out the dataset validator and giving us feedback.