Static code Analysis - text log - what severity levels are accepted / expected ?

[Josverl]

Im using PyRight for static code checking.

In the text formats described in the documentation
GCC (aka "GNU Emacs")

• file[:line[:column]]: type: message
  Visual Studio
• file[(line[,column])]: type: message

Pyright uses a slightly different log format, but it's not to difficult to transform with a regex to match.

The line format is clear , but not what the accepted values or interpretation of type is.

based on the below sample PyRight log

.\stubs\micropython-latest-docstubs\machine.py
  .\stubs\micropython-latest-docstubs\machine.py:221:37 - info: "IRQ_FALLING" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:575:59 - info: "IDLE" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:617:18 - info: "MSB" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:705:18 - warning: "MSB" is not defined (reportUndefinedVariable)
  .\stubs\micropython-latest-docstubs\machine.py:1039:50 - info: "IDLE" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:1068:28 - info: "PERIODIC" is unbound (reportUnboundVariable)
.\stubs\micropython-latest-docstubs\machine.pyi
  .\stubs\micropython-latest-docstubs\machine.pyi:32:62 - error: Non-default argument follows default argument

transformed to match the GCC format : (-replace ' - (info|warning|error):', ': $1:')

.\stubs\micropython-latest-docstubs\machine.py
  .\stubs\micropython-latest-docstubs\machine.py:221:37: info: "IRQ_FALLING" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:575:59: info: "IDLE" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:617:18: info: "MSB" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:705:18: warning: "MSB" is not defined (reportUndefinedVariable)
  .\stubs\micropython-latest-docstubs\machine.py:1039:50: info: "IDLE" is unbound (reportUnboundVariable)
  .\stubs\micropython-latest-docstubs\machine.py:1068:28: info: "PERIODIC" is unbound (reportUnboundVariable)
.\stubs\micropython-latest-docstubs\machine.pyi
  .\stubs\micropython-latest-docstubs\machine.pyi:32:62: error: Non-default argument follows default argument

I see a mapping of the log the Severity

• error --> High
• warning --> medium
• info --> No mapping ?

If I look in the metrics I see no 'Low' metrics captured, and I expected the info class to be seen as Low , similar as Error is seen as High .
Are info/low filtered out somewhere, if so how could I change that ?
If I want to see 'Low Severity' in the ; what keyword should I use ?

Datasource: /all-stubs/Static Analysis[sa_issues]

[ivailop]

Just to confirm, by PyRight you mean https://github.com/microsoft/pyright?

As you discovered its output requires minor adjustment (i.e. : vs -) to match the Testspace expected format. In addition, to properly process info entries, you would have to change them to information (full word).

---

We would consider, updating Testspace to support the PyRight unique output format. Meanwhile, do you thing it is appropriate to have PyRight fixing its output to (questionably) be more standard (widely accepted) format? I believe, the required changes are isolated to single function:

• - to : at https://github.com/microsoft/pyright/blob/1732614749ff77fab577ba6af02d92cb253882c5/packages/pyright-internal/src/pyright.ts#L748
• info to information at https://github.com/microsoft/pyright/blob/1732614749ff77fab577ba6af02d92cb253882c5/packages/pyright-internal/src/pyright.ts#L758

[Josverl]

Yes , that is right pyright,

I could creating an issue @pyright , ; but it would help make the case if there was clear description of the format on order to make the case for such a change. I searched for that before asking here, but could not find a clear description.
May I suggest that you add the accepted severity levels to the data format doc , possibly with a link to an external full spec .

In trying to make this work I found that a single space would throw off testspace from processing the log files.
perhaps tweaking a regex on the import mapping may work as well ,
for my project im fine with making the changes myself, now that I know how to make it work.

[ivailop]

Unfortunately, there are no formal definitions of the standard log formats, but we believe (based on all we have seen) the tools out there align with GCC/VisualStudio syntax - https://help.testspace.com/reference/data-formats/#text-log

I agree, info is a miss on our (Testspace) side and will try to address it. We would try to also handle the - (in addition to :)...

However, for the : vs -, if you wish to make a case @pyright here are few references:

• flake8 - https://flake8.pycqa.org/en/latest/internal/formatters.html#default-formatter
• pylint (default/parseable)- https://pylint.pycqa.org/en/latest/user_guide/output.html#custom-message-formats
• pycodestyle - https://pycodestyle.pycqa.org/en/latest/intro.html#example-usage-and-output

[Josverl]

Now that Pyright has changed to use the "information" label, and already pushed that to production , it would be good if you can find a way to accept the : vs -, or add functionality to ingest the pyright json format