Skip to content

Commit 637558a

Browse files
committed
Add prompt and command variants of infinite block test
Renamed the original infinite block test to clarify it uses a prompt action, and added a new command-based variant that uses `false` (always fails). Both tests verify the promise mechanism: - Infinite Block Prompt: Shows instructions, promise bypasses - Infinite Block Command: Runs failing command, promise skips execution Testing confirmed that promises work correctly for both action types - when a promise is provided, command-action rules are skipped entirely and the command never runs.
1 parent e059de8 commit 637558a

6 files changed

Lines changed: 150 additions & 45 deletions

File tree

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
name: "Manual Test: Infinite Block Command"
3+
trigger: manual_tests/test_infinite_block_command/test_infinite_block_command.py
4+
action:
5+
command: "false"
6+
run_for: each_match
7+
compare_to: prompt
8+
---
9+
10+
# Manual Test: Infinite Block Command (Promise Required)
11+
12+
This rule runs a command that ALWAYS FAILS (`false` returns exit code 1).
13+
14+
## Why this blocks
15+
16+
The command action always fails, creating an infinite block. The only way
17+
to proceed should be to provide a promise acknowledging that you understand
18+
the restriction.
19+
20+
## Expected behavior
21+
22+
If promises work correctly for command actions:
23+
- Without promise: Command runs, fails, blocks
24+
- With promise: Command is SKIPPED entirely, allows
25+
26+
If there's a bug:
27+
- The command will run and fail even when a promise is provided
28+
29+
## What to do
30+
31+
You MUST include the following promise tag in your response:
32+
33+
```
34+
<promise>Manual Test: Infinite Block Command</promise>
35+
```
36+
37+
## This tests
38+
39+
Whether the promise mechanism works for COMMAND-type rules. If a rule is
40+
promised, the command should not run at all - the rule should be skipped
41+
during evaluation.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
name: "Manual Test: Infinite Block Prompt"
3+
trigger: manual_tests/test_infinite_block_prompt/test_infinite_block_prompt.py
4+
compare_to: prompt
5+
---
6+
7+
# Manual Test: Infinite Block Prompt (Promise Required)
8+
9+
You edited `{trigger_files}` which triggers an infinite block.
10+
11+
## Why this blocks
12+
13+
This rule has NO safety file option and uses a PROMPT action. The only way
14+
to proceed is to provide a promise acknowledging that you understand the
15+
restriction.
16+
17+
## What to do
18+
19+
You MUST include the following promise tag in your response:
20+
21+
```
22+
<promise>Manual Test: Infinite Block Prompt</promise>
23+
```
24+
25+
This simulates scenarios where:
26+
- An operation requires explicit acknowledgment before proceeding
27+
- There is no alternative action that can suppress the rule
28+
- The agent must demonstrate understanding of the constraint
29+
30+
## This tests
31+
32+
The promise mechanism for PROMPT-type rules that cannot be satisfied by
33+
editing additional files. This is useful for enforcing policies where
34+
acknowledgment is the only valid response.

.deepwork/rules/manual-test-infinite-block.md

Lines changed: 0 additions & 33 deletions
This file was deleted.

manual_tests/README.md

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,8 @@ Each test has two cases: one where the rule SHOULD fire, and one where it should
3434
| **Pair Mode (reverse)** || Edit `_expected.md` only (should NOT fire) | Manual Test: Pair Mode |
3535
| **Command Action** | Edit `.txt` → log appended | — (always runs) | Manual Test: Command Action |
3636
| **Multi Safety** | Edit `.py` only | Edit `.py` AND any safety file | Manual Test: Multi Safety |
37-
| **Infinite Block** | Edit `.py` (always blocks) | Provide `<promise>` tag | Manual Test: Infinite Block |
37+
| **Infinite Block Prompt** | Edit `.py` (always blocks) | Provide `<promise>` tag | Manual Test: Infinite Block Prompt |
38+
| **Infinite Block Command** | Edit `.py` (command fails) | Provide `<promise>` tag | Manual Test: Infinite Block Command |
3839

3940
## Test Results Tracking
4041

@@ -46,7 +47,8 @@ Each test has two cases: one where the rule SHOULD fire, and one where it should
4647
| Pair Mode (reverse - expected only) |||
4748
| Command Action |||
4849
| Multi Safety |||
49-
| Infinite Block |||
50+
| Infinite Block Prompt |||
51+
| Infinite Block Command |||
5052

5153
## Test Folders
5254

@@ -57,7 +59,8 @@ Each test has two cases: one where the rule SHOULD fire, and one where it should
5759
| `test_pair_mode/` | Pair (Directional) | One-way: trigger requires expected, but not vice versa |
5860
| `test_command_action/` | Command Action | Automatically runs command on file change |
5961
| `test_multi_safety/` | Multiple Safety | Fires unless ANY of the safety files also edited |
60-
| `test_infinite_block/` | Infinite Block | Always blocks; only promise can bypass |
62+
| `test_infinite_block_prompt/` | Infinite Block (Prompt) | Always blocks with prompt; only promise can bypass |
63+
| `test_infinite_block_command/` | Infinite Block (Command) | Command always fails; tests if promise skips command |
6164

6265
## Corresponding Rules
6366

@@ -67,4 +70,5 @@ Rules are defined in `.deepwork/rules/`:
6770
- `manual-test-pair-mode.md`
6871
- `manual-test-command-action.md`
6972
- `manual-test-multi-safety.md`
70-
- `manual-test-infinite-block.md`
73+
- `manual-test-infinite-block-prompt.md`
74+
- `manual-test-infinite-block-command.md`
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
"""
2+
MANUAL TEST: Infinite Block Command Rule (Promise Required)
3+
4+
=== WHAT THIS TESTS ===
5+
Tests a COMMAND-type rule with a command that ALWAYS FAILS - it will ALWAYS
6+
block when the trigger file is edited. This tests whether the promise mechanism
7+
works for command-action rules.
8+
9+
This verifies:
10+
1. The rule correctly blocks when the file is edited (command fails)
11+
2. The promise mechanism should bypass the command entirely
12+
3. The promise must be in the exact format: <promise>Rule Name</promise>
13+
14+
=== TEST CASE 1: Rule SHOULD fire (command fails, infinite block) ===
15+
1. Edit this file (add a comment below the marker)
16+
2. Run: echo '{}' | python -m deepwork.hooks.rules_check
17+
3. Expected: Block with command error - the `false` command always fails
18+
19+
=== TEST CASE 2: Rule should NOT fire (promise provided) ===
20+
1. Edit this file (add a comment below the marker)
21+
2. Create a transcript with: <promise>Manual Test: Infinite Block Command</promise>
22+
3. Run the hook with the transcript
23+
4. Expected: Empty JSON {} (allow) - promise should bypass the command entirely
24+
25+
=== HOW TO TEST WITH PROMISE ===
26+
The promise must be in the conversation transcript. To test:
27+
28+
1. Create a temp transcript file with the promise:
29+
echo '{"role":"assistant","message":{"content":[{"type":"text","text":"<promise>Manual Test: Infinite Block Command</promise>"}]}}' > /tmp/transcript.jsonl
30+
31+
2. Run with transcript:
32+
echo '{"transcript_path":"/tmp/transcript.jsonl"}' | python -m deepwork.hooks.rules_check
33+
34+
3. Expected: {} (empty JSON = allow)
35+
If NOT empty: BUG - promises don't work for command-action rules!
36+
37+
=== RULE LOCATION ===
38+
.deepwork/rules/manual-test-infinite-block-command.md
39+
40+
=== KEY DIFFERENCE FROM PROMPT VERSION ===
41+
- Prompt version: Shows instructions, agent must respond with promise
42+
- Command version: Runs a command that always fails, promise should skip it
43+
44+
If the promise mechanism doesn't work for command rules, this test will reveal
45+
that bug - the command will still run and fail even when a promise is provided.
46+
"""
47+
48+
49+
def restricted_command_operation():
50+
"""An operation that requires explicit acknowledgment to proceed."""
51+
return "This operation uses a command that always fails"
52+
53+
54+
# Edit below this line to trigger the rule
55+
# -------------------------------------------
56+
# Test edit for command block

manual_tests/test_infinite_block/test_infinite_block.py renamed to manual_tests/test_infinite_block_prompt/test_infinite_block_prompt.py

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
"""
2-
MANUAL TEST: Infinite Block Rule (Promise Required)
2+
MANUAL TEST: Infinite Block Prompt Rule (Promise Required)
33
44
=== WHAT THIS TESTS ===
5-
Tests a rule with NO safety file option - it will ALWAYS block when the
6-
trigger file is edited. The only way to proceed is to provide a promise
7-
in the correct format.
5+
Tests a PROMPT-type rule with NO safety file option - it will ALWAYS block
6+
when the trigger file is edited. The only way to proceed is to provide a
7+
promise in the correct format.
88
99
This verifies:
1010
1. The rule correctly blocks when the file is edited
@@ -14,34 +14,37 @@
1414
=== TEST CASE 1: Rule SHOULD fire (infinite block) ===
1515
1. Edit this file (add a comment below the marker)
1616
2. Run: echo '{}' | python -m deepwork.hooks.rules_check
17-
3. Expected: "Manual Test: Infinite Block" appears in output with decision="block"
17+
3. Expected: "Manual Test: Infinite Block Prompt" appears in output with decision="block"
1818
4. The block message should explain that a promise is required
1919
2020
=== TEST CASE 2: Rule should NOT fire (promise provided) ===
2121
1. Edit this file (add a comment below the marker)
22-
2. Create a transcript with: <promise>Manual Test: Infinite Block</promise>
22+
2. Create a transcript with: <promise>Manual Test: Infinite Block Prompt</promise>
2323
3. Run the hook with the transcript
2424
4. Expected: Empty JSON {} (allow) - promise bypasses the block
2525
2626
=== HOW TO TEST WITH PROMISE ===
2727
The promise must be in the conversation transcript. To test:
2828
2929
1. Create a temp transcript file with the promise:
30-
echo '{"role":"assistant","message":{"content":[{"type":"text","text":"<promise>Manual Test: Infinite Block</promise>"}]}}' > /tmp/transcript.jsonl
30+
echo '{"role":"assistant","message":{"content":[{"type":"text","text":"<promise>Manual Test: Infinite Block Prompt</promise>"}]}}' > /tmp/transcript.jsonl
3131
3232
2. Run with transcript:
3333
echo '{"transcript_path":"/tmp/transcript.jsonl"}' | python -m deepwork.hooks.rules_check
3434
3535
3. Expected: {} (empty JSON = allow)
3636
3737
=== RULE LOCATION ===
38-
.deepwork/rules/manual-test-infinite-block.md
38+
.deepwork/rules/manual-test-infinite-block-prompt.md
3939
4040
=== KEY DIFFERENCE FROM OTHER TESTS ===
4141
Other tests have a "safety" file that can be edited to suppress the rule.
4242
This test has NO safety option - the ONLY way to proceed is with a promise.
4343
This simulates scenarios where the agent must explicitly acknowledge a
4444
constraint before proceeding.
45+
46+
=== COMPARISON WITH COMMAND VERSION ===
47+
See test_infinite_block_command/ for the command-action version of this test.
4548
"""
4649

4750

0 commit comments

Comments
 (0)