chroot substitution is inconsistent with experimental_test_shell_command

[jcrobak]

Describe the bug
chroot substitution is inconsistent with experimental_test_shell_command (and others?).

Specifically, I noticed that saying echo "{chroot}" works as expected by echo "{chroot}/foo" does not.

Pants version
2.24.1

OS
MacOS

Additional info

Here's a self-contained script to repro the problem:

#!/bin/bash

# Create the pants.toml file
cat << 'EOF' > pants.toml
[GLOBAL]
pants_version = "2.24.1"

backend_packages = [
      "pants.backend.shell",
]
EOF

# Create the BUILD file
cat << 'EOF' > BUILD
experimental_test_shell_command(
    name="test",
    command="""
        echo "starting test..."
        echo "{chroot}"
        echo "{chroot}/foo"
        echo "...test complete"
        exit 1
    """,
    log_output=True,
)
EOF

pants test ::

Example output:

09:39:35.57 [INFO] Initializing scheduler...
09:39:35.59 [INFO] Initializing Nailgun pool for 28 processes...
09:39:35.68 [INFO] Scheduler initialized.
09:39:35.87 [INFO] Completed: Running the `experimental_test_shell_command` at `//:test`
09:39:35.88 [ERROR] Completed: Test with shell command - //:test - failed (exit code 1).
starting test...
/private/var/folders/mm/n9m9_2sd77b4gqm1938rhx_m0000gn/T/pants-sandbox-6hlDar
foo
...test complete

The expected output should look like:

starting test...
/private/var/folders/mm/n9m9_2sd77b4gqm1938rhx_m0000gn/T/pants-sandbox-6hlDar
/private/var/folders/mm/n9m9_2sd77b4gqm1938rhx_m0000gn/T/pants-sandbox-6hlDar/foo
...test complete

[tdyas]

I put a space between the {chroot} and the /foo parts and the second substitution worked as expected. I also changed the / to a - and the second substitution also works as expected. So the / is somehow being interpreted weirdly by some component of the system.

[tdyas]

The issue seems to be in the display of the test output and not with the {chroot} substitution itself. When I run under test --debug and manually inspect the TestResult, the expected {chroot} substitutions are there. The test goal applies an "output simplifier" usually which strips any path-like string in the output which looks like a sandbox path. See

pants/src/python/pants/util/strutil.py

Line 147 in b7f08e9

class Simplifier:

and its call to the strip_v2_chroot_path function.

So the question really becomes one of how do we disable the output simplifier or document the behavior.

[tdyas]

Relevant code in core test goal:

pants/src/python/pants/core/goals/test.py

Line 245 in bdecc47

def stdout_simplified_str(self) -> str:

[tdyas]

@jcrobak: Would you want the ability to disable the output simplifier?

[jcrobak]

@tdyas from what I can tell, this output is only shown when the command fails... and in my case I was trying to debug. So it was pretty confusing, since I was trying to determine what exact steps my test was executing. I'm not sure if having the ability to remove the output simplifier would help—would I have know to look for that setting?

I'm also not sure if it makes sense to turn the output simplifier on/off based on whether or not the test is passing because I could see that leading to a lot of confusion.

Perhaps the fix here is better documentation?