fix: tagging JSON 파싱 prefix 흡수 (assistant prefill + 슬라이스 fallback)#19
Merged
Conversation
gyesswhat
added
fix
📝 Walkthrough🚀 WalkthroughHaiku 4.5 응답의 도입어 prefix로 인한 JSON 파싱 실패 문제를 해결하기 위해 LLM 프롬프팅 및 파싱 파이프라인을 재구성합니다. Assistant prefill 메시지 주입, 강화된 JSON 추출 함수, 토큰 한도 증가를 조합하여 안정성을 개선하고, 이를 검증하는 단위/통합 테스트를 확장합니다. 📋 Changesv2_postscore JSON 파싱 강화
🔄 Sequence DiagramsequenceDiagram
participant Caller as 호출자
participant run as run()
participant LLM as Claude Haiku 4.5
participant extract as _extract_json_payload
participant validate as _parse_and_validate
Caller->>run: tag_request 호출
rect rgba(100, 150, 200, 0.5)
Note over run,validate: 1차 LLM 호출
run->>LLM: messages=[user, assistant("{")]
LLM-->>run: "detailTags": [...] (도입어 생략)
run->>run: first_raw = "{" + extract_text(...)
run->>validate: "{detailTags": [...]} 전달
validate->>extract: JSON 추출 시도
extract-->>validate: 정제된 JSON object
validate-->>run: 검증 성공
end
rect rgba(200, 100, 100, 0.5)
Note over run,validate: 검증 실패 시 2차 재시도
run->>LLM: messages=[user, assistant("{")]
LLM-->>run: "detailTags": [...] (또 도입어)
run->>validate: "{detailTags": [...]} 전달
validate-->>run: 검증 성공 또는 최종 실패
end
run-->>Caller: tag_result 반환
📊 Code Review Effort🎯 3 (Moderate) | ⏱️ ~25분 이유:
🔗 Possibly Related PRs
🏷️ Suggested Labels
👥 Suggested Reviewers
🎭 Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsStopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/test_tagging.py (1)
275-317: ⚡ Quick win🧪 JSON 잘림(닫는
}누락) 실패 케이스도 1개 추가하면 더 안전합니다.현재 성공 케이스 커버는 충분히 좋습니다. 여기에
max_tokens잘림처럼 불완전 JSON 입력이 들어왔을 때_v2_parse_and_validate가TaggingValidationError로 안정적으로 실패하는 경계 테스트를 하나 고정해두면 회귀 방지 효과가 큽니다.예시 테스트
+def test_v2_postscore_parse_and_validate_rejects_truncated_payload() -> None: + raw = '다음과 같습니다: {"detailTags": ["`#원인분석`","`#검증및테스트`"' + with pytest.raises(TaggingValidationError, match="JSON 파싱 실패"): + _v2_parse_and_validate(raw)As per coding guidelines "tests/**: 테스트 코드를 검토한다. - 해피 패스뿐 아니라 경계값·실패 케이스도 커버하는지 확인".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_tagging.py` around lines 275 - 317, Add a negative boundary test that passes an incomplete/truncated JSON (e.g. missing the closing brace) through _v2_parse_and_validate and assert it raises TaggingValidationError; this verifies the parser fails reliably on max_tokens-like JSON truncation. Put the new test alongside existing tests (near test_v2_postscore_parse_and_validate_handles_intro_prefix) and use a raw string that mirrors real responses (e.g. prefix like "다음과 같습니다: " + '{"detailTags": ["`#원인분석`","`#검증및테스트`"' ) so _v2_extract_json_payload is exercised before _v2_parse_and_validate raises TaggingValidationError.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/test_tagging.py`:
- Around line 275-317: Add a negative boundary test that passes an
incomplete/truncated JSON (e.g. missing the closing brace) through
_v2_parse_and_validate and assert it raises TaggingValidationError; this
verifies the parser fails reliably on max_tokens-like JSON truncation. Put the
new test alongside existing tests (near
test_v2_postscore_parse_and_validate_handles_intro_prefix) and use a raw string
that mirrors real responses (e.g. prefix like "다음과 같습니다: " + '{"detailTags":
["`#원인분석`","`#검증및테스트`"' ) so _v2_extract_json_payload is exercised before
_v2_parse_and_validate raises TaggingValidationError.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 8a340da6-d6df-48b0-8476-b93fe73d2c0a
📒 Files selected for processing (2)
app/services/tagging/v2_postscore.pytests/test_tagging.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
요약
char 0 unexpectedJSON 파싱 실패가 24h 164회 발생하던 운영 이슈를 assistant prefill +{...}슬라이스로 흡수.변경 사항
코드
_extract_json_payload신규 — 코드펜스 strip 후 첫{~ 마지막}슬라이스. 도입어·꼬리말 흡수._parse_and_validate가_strip_code_fence→_extract_json_payload호출.run의 1차/2차 호출 모두 messages 마지막에{"role": "assistant", "content": "{"}prefill 추가. 응답 처리 시_ASSISTANT_PREFILL + raw로 합쳐 파싱 → 첫 글자{100% 보장._MAX_TOKENS250 → 350 — prefix 흡수 마진 + max_tokens 잘림 1건 (운영 실측) 방지.tagging.validation_failed로그에raw_head(응답 앞 80자) 추가 — 패치 후 재발생 모니터링용. STAR 본문(PII) 미포함이라 안전. 안정 확인되면 별도 PR 로 제거 예정.테스트
_extract_json_payload6 케이스 (clean / 한국어 prefix / markdown prefix+suffix / 공백 / 코드펜스 / 브레이스 없음 passthrough)._parse_and_validate_handles_intro_prefix— 도입어 응답 e2e 파싱.{제거 (헤더_HAPPY_LLM_RAW등). 코멘트로 의미 명시.테스트
uv run pytest→ 77 passeduv run ruff check app/ tests/→ All checks passeduv run ruff format --check app/ tests/→ 변경 없음uv run mypy app→ pre-existing anthropic stubs 누락 1건 (본 변경과 무관)스크린샷/로그
운영 실측 (stg
groute-aidocker logs, 2026-05-28 기준):→ 도입어/markdown prefix 가 원인. 본 PR 머지 + 배포 후
tagging.validation_failed빈도 추이 모니터링.롤백/리스크
_extract_json_payload의{~}슬라이스는 본문에 중첩 brace ({ ... { ... } ... }) 가 들어와도 가장 바깥{와 가장 마지막}로 잘 잡힘. detailTags 응답 스키마는 nested object 없으므로 안전._MAX_TOKENS350 으로 +40% 늘어남 → 비용 영향 미미 (output 단가 $5/Mtok, 평균 응답 ~50 tok 이라 실질 인상은 거의 0).관련 이슈
Closes #18
Summary by CodeRabbit
릴리스 노트
버그 수정
테스트