Skip to content

Fix: append_target_turn accepts raw strings, causing AttributeError in multi-turn attacks - #222

Open
vje013 wants to merge 2 commits into
confident-ai:mainfrom
vje013:fix/append-target-turn-type-safety
Open

Fix: append_target_turn accepts raw strings, causing AttributeError in multi-turn attacks#222
vje013 wants to merge 2 commits into
confident-ai:mainfrom
vje013:fix/append-target-turn-type-safety

Conversation

@vje013

@vje013 vje013 commented Apr 27, 2026

Copy link
Copy Markdown

Title
Fix: append_target_turn accepts raw strings, causing AttributeError: 'str' object has no attribute 'role' in multi-turn attacks

Summary
This PR fixes a bug where LinearJailbreaking and other multi-turn attacks crash when append_target_turn() inserts raw strings into the turn history. The function's signature declares target_response: RTTurn, but every caller in the library passes the raw string returned by model_callback(...). Downstream code (e.g. JailBreakingTemplate.improvement_prompt) then iterates the history expecting .role and .content attributes and fails with:
AttributeError: 'str' object has no attribute 'role'
This PR aligns runtime behavior with the declared contract by normalizing the input inside append_target_turn, ensuring the turn history always contains RTTurn objects.

Issue Description
Root Cause
append_target_turn() is typed to receive an RTTurn but is universally called with a raw string:

deepteam/attacks/multi_turn/utils.py

def append_target_turn(
turns: List[RTTurn], target_response: RTTurn, turn_level_attack: str = None
):
if turn_level_attack:
target_response.turn_level_attack = turn_level_attack
turns.append(target_response)

deepteam/attacks/multi_turn/linear_jailbreaking/linear_jailbreaking.py

target_response = model_callback(current_attack, turns)
append_target_turn(turns, target_response) # target_response is a str
Later, JailBreakingTemplate.improvement_prompt() iterates the history:
for turn in turn_history:
if turn.role == "user":
...
This fails when turn is a string.

Impact
All multi-turn attacks (LinearJailbreaking, SequentialJailbreak, etc.) crash at the first appended assistant response.
A second latent AttributeError exists on the turn_level_attack branch: target_response.turn_level_attack = turn_level_attack runs before any type check, so it would also fail on a string. The original traceback didn't surface it only because the random turn_level_attacks branch didn't fire.
Users cannot run red-team evaluations unless they manually wrap model responses in RTTurn inside their model_callback.
The error is non-obvious — the traceback points at a template file, several layers away from the actual append site.

Reproduction Steps
Minimal reproducible example:
from deepteam.test_case import RTTurn
from deepteam.attacks.multi_turn.utils import append_target_turn
from deepteam.attacks.multi_turn.linear_jailbreaking.template import JailBreakingTemplate

turns = [RTTurn(role="user", content="test")]
response = "hello world" # model_callback returns a str
append_target_turn(turns, response)

JailBreakingTemplate.improvement_prompt(turns, "feedback")

AttributeError: 'str' object has no attribute 'role'

The same crash reproduces end-to-end by running any LinearJailbreaking attack with a model_callback that returns a string (which is the documented and idiomatic return type).

Proposed Fix
Normalize inside append_target_turn so the turn history is guaranteed to contain RTTurn objects, regardless of what model_callback returns. The wrapping happens before the turn_level_attack assignment, fixing both crashes in one change.

Patch
def append_target_turn(

  • turns: List[RTTurn], target_response: RTTurn, turn_level_attack: str = None
  • turns: List[RTTurn],
  • target_response, # str | RTTurn | other
  • turn_level_attack: str = None,
    ):
  • if turn_level_attack:
  •    target_response.turn_level_attack = turn_level_attack
    
  • turns.append(target_response)
  • Normalize to RTTurn so the history is always typed consistently.

  • if isinstance(target_response, RTTurn):
  •    turn = target_response
    
  • elif isinstance(target_response, str):
  •    turn = RTTurn(role="assistant", content=target_response)
    
  • else:
  •    # Fallback for any other model-specific return types
    
  •    turn = RTTurn(role="assistant", content=str(target_response))
    
  • if turn_level_attack:
  •    turn.turn_level_attack = turn_level_attack
    
  • turns.append(turn)
    This guarantees type safety without requiring users to modify their model_callback.

Alternative Considered
Tightening the callers instead — wrapping the response into an RTTurn at every call site in linear_jailbreaking.py, sequential_jailbreak.py, etc. Rejected because (a) it touches more files, (b) it leaves the utility's type contract unenforced, and (c) it doesn't help third-party attacks that depend on this helper. Happy to switch to this approach if maintainers prefer it.

Additional Improvements
Added a unit test asserting append_target_turn() produces RTTurn objects for str, RTTurn, and arbitrary inputs.
Added a test covering the turn_level_attack branch with a string input (regression for the second latent crash).
Updated the docstring to clarify the accepted input types.

Why This Fix Is Safe
No changes to attack logic or judge/improvement prompts.
No changes to model behavior or output content.
Only enforces a data-structure invariant that downstream code already assumed.
Backwards compatible: callers that already pass an RTTurn get identical behavior.

Checklist
[x] Bug reproduced
[x] Fix implemented
[x] Tests added (including regression for the turn_level_attack branch)
[x] No breaking API changes
[x] Multi-turn attacks now run end-to-end without errors

vje013 added 2 commits April 27, 2026 13:03
Wrap str and other non-RTTurn inputs into RTTurn(role="assistant", ...)
  before appending and before any attribute assignment. Fixes
  AttributeError in downstream template iteration and a latent
  AttributeError on the turn_level_attack branch.
Add regression tests for append_target_turn function to ensure type safety and proper handling of different response types.
@vercel

vercel Bot commented Apr 27, 2026

Copy link
Copy Markdown

@vje013 is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant