Show error message with details about step validation by jensakejohansson · Pull Request #2837 · getgauge/gauge

jensakejohansson · 2026-03-31T22:01:39Z

When a runner (e.g. gauge-dotnet) detects a duplicate step implementation, it returns a StepValidateResponse with both an errorType (DUPLICATE_STEP_IMPLEMENTATION) and an errorMessage containing details like file paths and line numbers of the conflicting implementations.

However, Gauge ignores the errorMessage field, the displayed message is only derived from the enum name DUPLICATE_STEP_IMPLEMENTATION → "Duplicate step implementation", discarding any additional context the runner provided.

Fix:

After deriving the base message from the error type append the runner's errorMessage if it is non-empty.
This is backwards-compatible with all runners. errorMessage is a string field in the protobuf contract — if a runner doesn't set it, GetErrorMessage() returns "" and the behavior is unchanged.

Example:

Before:
Duplicate step implementation

Now:
Duplicate step implementation
Step : Click {}
ActionsA.Click in ActionsA.cs:15
ActionsB.Click in ActionsB.cs:42

…n - show it. Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

jensakejohansson · 2026-04-01T00:24:39Z

Hopefully fixes failing tests: getgauge/gauge-tests#153

The errors messages differs between the language runners and might need to be improved in places, but I still believe it's a good idea to show them in the IDE. They can always be improved later on.

chadlwilson · 2026-04-01T04:08:27Z

I've merged the test change just to move forward with the tests here but I wonder

if it's a good idea to have these messages be multi line now (might depend on IDE how that is rendered and make the console logs harder to parse?)

should the detail suffix the step name rather than prefix it? Perhaps like

foo.spec:3 Step implementation not found => 'my step'
    detail: my err msg

Do you happen to know if there are other places currently where such messages can be multi-lined?

Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

jensakejohansson · 2026-04-01T11:08:47Z

Yea, I revised this as follows:

Insight: Message() is only used by the IDE LSP diagnostics path, while Error() is used by console, reports, plugins, skip errors, and all other display paths. So what about we add the detail to Message() only?

That would address the concern about console output, which I believe to be important. The output may become too noisy. Especially since, at first glance, it doesn't seems to have been a coherent approach to what message content different language implementations should provide.

Also, no need to update any tests (unit or the integration tests).

Still, when developing in an IDE you can get some extra useful information from the runner. Yes, one can argue that error messages becomes inconsistent across "environments" - but to me it makes some sense. Any opinions?

jensakejohansson · 2026-04-01T12:37:29Z

getgauge/gauge-lsp-tests#131

A PR that should make the LSP tests pass. Check only the first line in message - indicating the error type - but let details-part slip through....

P.S
Tried locally, but test execution was for some reason not really reliable on my machine (slow and got occasional timeouts!).

chadlwilson · 2026-04-01T13:05:15Z

P.S Tried locally, but test execution was for some reason not really reliable on my machine (slow and got occasional timeouts!).

Assuming you’re on Windows the LSP tests are totally unusable on Windows for some reason. I have no idea why. Windows is slow, yes, but they take 5-6 times longer than Linux or MacOS which is insane.

github-actions · 2026-04-01T13:31:16Z

Benchmark Results

java_gradle_parallel.csv

Commit	CPU	Memory	Time
`4806ca1`	122%	546804	0:42.68
`72ae204`	113%	556332	0:44.66
`a120821`	120%	544140	0:41.52
`7471510`	5%	116208	0:42.60

java_simple_parallel.csv

Commit	CPU	Memory	Time
`4806ca1`	23%	68096	0:24.86
`72ae204`	21%	68512	0:26.50
`a120821`	21%	68436	0:27.03
`7471510`	22%	68432	0:24.73

java_gradle_multithreaded.csv

Commit	CPU	Memory	Time
`4806ca1`	200%	536500	0:25.63
`72ae204`	201%	551452	0:23.86
`a120821`	204%	560352	0:24.87
`7471510`	197%	551096	0:25.25

java_simple_multithreaded.csv

Commit	CPU	Memory	Time
`4806ca1`	35%	68112	0:11.73
`72ae204`	33%	68596	0:11.90
`a120821`	33%	68160	0:11.72
`7471510`	33%	68356	0:11.44

java_gradle_serial.csv

Commit	CPU	Memory	Time
`4806ca1`	194%	551996	0:27.21
`72ae204`	199%	556468	0:26.36
`a120821`	202%	534920	0:26.03
`7471510`	200%	545456	0:26.37

java_simple_serial.csv

Commit	CPU	Memory	Time
`4806ca1`	42%	66320	0:12.22
`72ae204`	58%	68028	0:11.56
`a120821`	43%	68256	0:12.62
`7471510`	52%	68384	0:11.84

java_maven_serial.csv

Commit	CPU	Memory	Time
`4806ca1`	69%	164512	0:20.22
`72ae204`	74%	160956	0:18.22
`a120821`	75%	164132	0:18.62
`7471510`	62%	163072	0:19.14

java_maven_parallel.csv

Commit	CPU	Memory	Time
`4806ca1`	33%	161880	0:37.63
`72ae204`	35%	158404	0:32.32
`a120821`	37%	162576	0:32.17
`7471510`	34%	164820	0:37.95

java_maven_multithreaded.csv

Commit	CPU	Memory	Time
`4806ca1`	51%	161112	0:19.41
`72ae204`	52%	164904	0:20.86
`a120821`	63%	163432	0:17.44
`7471510`	61%	162560	0:17.62

Notes

The results above are generated by running against seed projects in https://github.com/getgauge/gauge-benchmark
These results are not persisted, but on merging to master the benchmark will be rerun.
These benchmark are run in Github Actions' agents, which are virtualized. Results are not to be taken as actual benchmarks.Rather, these are indicative numbers and make sense for comparison.

See Workflow log for more details.

chadlwilson · 2026-04-01T15:13:41Z

Yea, I revised this as follows:

Insight: Message() is only used by the IDE LSP diagnostics path, while Error() is used by console, reports, plugins, skip errors, and all other display paths. So what about we add the detail to Message() only?

That would address the concern about console output, which I believe to be important. The output may become too noisy. Especially since, at first glance, it doesn't seems to have been a coherent approach to what message content different language implementations should provide.

Also, no need to update any tests (unit or the integration tests).

Still, when developing in an IDE you can get some extra useful information from the runner. Yes, one can argue that error messages becomes inconsistent across "environments" - but to me it makes some sense. Any opinions?

im not sure how important the console log format is, since i dont tend to use the validate workflow in normal operation

Do you still get the extended detail information in the lsp runner and gauge log files when working with an IDE even if we don’t include in the console output? If so, that may be sufficient.

Other than that @sriv or @zabil may be able to give some insight into general logging methodology, and the wisdom of including line breaks in either messages returned for console log or the IDE LsP responses.

jensakejohansson · 2026-04-01T18:49:43Z

Do you still get the extended detail information in the lsp runner and gauge log files when working with an IDE even if we don’t include in the console output? If so, that may be sufficient.

That can be achieved by logging in the language runner. I added this to getgauge/gauge-dotnet#287

Logs at debug-level in the runner it will not be routed to the console out by default, since default log-level is set to info. If you run Gauge CLI with log-level explicitly set to debug it will of course show up in console together with a lot of other logging that is normally not shown.

Example of output:

gauge.log example:

01-04-2026 20:39:36.877 [Gauge] [DEBUG] Validation started.
01-04-2026 20:39:36.975 [dotnet] [DEBUG] Duplicate step implementation found for: Navigate to url {}
01-04-2026 20:39:36.975 [dotnet] [DEBUG] Duplicate: GenericBrowserActions.NavigateToApplicationUrl in c:\gauge\Actions\BrowserActions.cs:28
01-04-2026 20:39:36.975 [dotnet] [DEBUG] Duplicate: GenericBrowserActions.NavigateToApplicationUrl2 in c:\gauge\Actions\BrowserActions.cs:36
01-04-2026 20:39:36.999 [Gauge] [ERROR] [ValidationError] c:\gauge\concepts\test.cpt:8 Duplicate step implementation => 'Navigate to url "http://www.test.com"'
01-04-2026 20:39:36.999 [Gauge] [ERROR] [ValidationError] c:\gauge\specs\test.spec:6 Duplicate step implementation => 'Navigate to url "http://www.gauge.org"'

lsp.log example:

01-04-2026 20:41:36.356 [Gauge] [DEBUG] Validation started.
01-04-2026 20:41:36.370 [dotnet] [DEBUG] Duplicate step implementation found for: Navigate to url {}
01-04-2026 20:41:36.370 [dotnet] [DEBUG] Duplicate: GenericBrowserActions.NavigateToApplicationUrl in c:\gauge\Actions\BrowserActions.cs:28
01-04-2026 20:41:36.370 [dotnet] [DEBUG] Duplicate: GenericBrowserActions.NavigateToApplicationUrl2 in c:\gauge\Actions\BrowserActions.cs:36

console (at default log-level):

[ValidationError] C:\Users\JensJ\Projects\ReLoadTT_CSharp\csharp\gauge\Applications\NopCommerce\concepts\nopCommerce.cpt:8 Duplicate step implementation => 'Navigate to url "http://www.test.com"'
[ValidationError] C:\Users\JensJ\Projects\ReLoadTT_CSharp\csharp\gauge\Applications\NopCommerce_Playwright\specs\nopCommerce.spec:6 Duplicate step implementation => 'Navigate to url "http://www.gauge.org"'

And of course message to IDE.

jensakejohansson added 2 commits March 31, 2026 22:48

If runner provides an error message with details about step validatio…

0a25181

…n - show it. Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

Updated tests now that error message is appened.

affdbb4

Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

This was referenced Mar 31, 2026

Guards all StepRegistry mutations to make them thread-safe. getgauge/gauge-dotnet#287

Open

Only validate keyword words in step validations messages getgauge/gauge-tests#153

Merged

Add runner error details to IDE messages only.

1ac35c8

Signed-off-by: jensakejohansson <jens.johansson@systemverification.com>

jensakejohansson mentioned this pull request Apr 1, 2026

Validate only first line in message from runner, not details. getgauge/gauge-lsp-tests#131

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show error message with details about step validation#2837

Show error message with details about step validation#2837
jensakejohansson wants to merge 3 commits into
getgauge:masterfrom
system-verification:feature/show-step-validation-message-from-runner

jensakejohansson commented Mar 31, 2026 •

edited

Loading

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

chadlwilson commented Apr 1, 2026 •

edited

Loading

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

chadlwilson commented Apr 1, 2026

Uh oh!

github-actions Bot commented Apr 1, 2026

Uh oh!

chadlwilson commented Apr 1, 2026

Uh oh!

jensakejohansson commented Apr 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Conversation

jensakejohansson commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

chadlwilson commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

jensakejohansson commented Apr 1, 2026

Uh oh!

chadlwilson commented Apr 1, 2026

Uh oh!

github-actions Bot commented Apr 1, 2026

Benchmark Results

Notes

Uh oh!

chadlwilson commented Apr 1, 2026

Uh oh!

jensakejohansson commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

jensakejohansson commented Mar 31, 2026 •

edited

Loading

chadlwilson commented Apr 1, 2026 •

edited

Loading

jensakejohansson commented Apr 1, 2026 •

edited

Loading