Tweak build and test instructions in AGENTS #13251
Tweak build and test instructions in AGENTS
#13251rainersigwald wants to merge 2 commits intomainfrom
Conversation
Some of the instructions were stale; some things seem to be base model problems like not understanding how to use file loggers.
There was a problem hiding this comment.
Pull request overview
Updates AGENTS.md build/test guidance to reduce stale instructions and improve troubleshooting workflows for building and running tests in this repo.
Changes:
- Adds guidance to avoid piping MSBuild output and suggests using a file logger instead.
- Adds an “Individual projects” build command and tweaks bootstrap-environment wording.
- Simplifies per-project test commands and adds an example for running a single test with
--filter.
| @@ -165,10 +146,11 @@ dotnet test src/Framework.UnitTests/Microsoft.Build.Framework.UnitTests.csproj | |||
| ./build.sh --test | |||
|
|
|||
| # Individual test project (recommended): | |||
| source artifacts/sdk-build-env.sh | |||
| dotnet test src/Framework.UnitTests/Microsoft.Build.Framework.UnitTests.csproj | |||
| ``` | |||
There was a problem hiding this comment.
The guidance here says to run dotnet test directly, but other repo docs still instruct activating the bootstrap environment first (e.g. documentation/wiki/Building-Testing-and-Debugging-on-.Net-Core-MSBuild.md shows running artifacts\msbuild-build-env.bat before dotnet test). Either re-add the environment-setup step here or clarify when it’s safe to run dotnet test without the bootstrap environment to avoid inconsistent instructions and potential SDK/MSBuild resolution issues.
| ## Building | ||
|
|
||
| **CRITICAL**: Never build with just `dotnet build MSBuild.slnx` or `dotnet build src/.../Project.csproj`. Always use the build scripts. | ||
| NEVER pipe MSBuild output to a file. If you want a list of errors, add `-flp:"v=q;LogFile=ErrorsAndWarnings.log"` to the arguments. |
There was a problem hiding this comment.
This suggests -flp:"v=q;LogFile=ErrorsAndWarnings.log" will produce a “list of errors”, but at quiet verbosity the file will still include warnings and some summary output. Consider rewording to “errors/warnings (plus summary)”, or explicitly document the exact logger parameters you want people to use.
| NEVER pipe MSBuild output to a file. If you want a list of errors, add `-flp:"v=q;LogFile=ErrorsAndWarnings.log"` to the arguments. | |
| NEVER pipe MSBuild output to a file. If you want to capture errors and warnings (plus summary) to a file, add `-flp:"v=q;LogFile=ErrorsAndWarnings.log"` to the arguments. |
There was a problem hiding this comment.
Reading through the hallucinations here, adding a "warnings are errors, don't introduce any" line.
| ## Building | ||
|
|
||
| **CRITICAL**: Never build with just `dotnet build MSBuild.slnx` or `dotnet build src/.../Project.csproj`. Always use the build scripts. | ||
| NEVER pipe MSBuild output to a file. If you want a list of errors, add `-flp:"v=q;LogFile=ErrorsAndWarnings.log"` to the arguments. |
There was a problem hiding this comment.
one thing we could instead suggest is always creating a binlog and using my MCP to get good data from the binlog
There was a problem hiding this comment.
I don't think this is the right strategy. I don't think we should advise using binlogs for "normal build errors"--our text log errors aren't bad, and they're well represented in the training data.
For any mildly complicated build-level investigation we should absolutely encourage the use of binlogs but I'm trying to correct the bad pattern I've seen of the agent doing dotnet build --verbosity quiet | tail -20, which works but is super ugly.
|
|
||
| Build individual projects with `dotnet msbuild {path/to/project.csproj} -v:q`. | ||
|
|
||
| ### Whole-repo Build Commands |
There was a problem hiding this comment.
I'd add "Do not use runTask tool to run those commands. Run them in blocking way."
The duration info in the table migh indirectly guide agents to use 'runTask' tool to run this async and periodicaly observe (the reason for why it might be cancelling it at times).
There was a problem hiding this comment.
What's the pros/cons there? Naïvely I would expect that what you're describing would be a good way to run build.sh, and possibly cancel it if there are early nonfatal errors? That feels like "how I do it as a human" . . .
Removes some mentions of `msbuild-build-env.{sh,bat}`, which should NOT be used when driving a build from the outside.
Some of the instructions were stale; some things seem to be base model problems like not understanding how to use file loggers.