Incorrect Inline Code Suggestion Formatting in Azure DevOps

[spanwair]

Git provider

Github Cloud

System Info

• PR-Agent Version: latest (Docker image: codiumai/pr-agent:latest)

• Runtime Environment: Azure DevOps Pipeline (Linux container job)

• Container OS: Debian-based (from PR-Agent image)

• Shell: bash

• Host Agent: Azure DevOps SHARED-BUILD-NIX pool

• Pipeline Type: YAML pipeline

• Git Provider: Azure DevOps (git_provider = "azure")

• Comment System: Azure DevOps PR Comments & Code Suggestions API

• Configuration File: .pr_agent.toml stored at repo root

• Trigger: Pull Request (Build.Reason == 'PullRequest')

• Commands Executed:

  • pr-agent describe
  • pr-agent review
  • pr-agent improve --pr_code_suggestions.committable_code_suggestions=true

• Environment Variables Provided:

  • azure_devops__pat
  • openai__key
  • Required Azure DevOps vars:
    SYSTEM_COLLECTIONURI, SYSTEM_TEAMPROJECT, BUILD_REPOSITORY_NAME, SYSTEM_PULLREQUEST_PULLREQUESTID

Bug details

Qodo’s inline code suggestions in Azure DevOps PRs are being rendered incorrectly.
Suggested patches are merged into one line, duplicated, or malformed — making the suggestions unusable and impossible to apply cleanly.

This happens even when configuration options related to suggestion filtering, chunk size, and formatting are tuned heavily. It looks like the formatting engine that generates diff suggestions is breaking the structure of multi-line changes.

---

Screenshots / Examples

Example 1 – Incorrectly Concatenated Suggestion Lines

In the example below (Image #1), the green suggestion block incorrectly merges two different suggestions into a single line:

committable_code_suggestions = truecommitable_code_suggestions = true

Only this should be shown:

committable_code_suggestions = true

Example 2 – Broken multi-line shell script suggestions

Another issue occurs with pipeline shell scripts. Qodo collapses and joins lines that should be separated, for example:

set -euo pipefail ... where set -euo pipefail should be on another line.

Instead of producing:

set -euo pipefail
echo "Running Qodo Merge on PR"

The suggestion output merges unrelated chunks into a single malformed block.

Example 3 – Suggestion blocks containing unrelated suggestions

Even when only one suggestion should appear, Qodo outputs something like:

echo "Running PR-Agent commands..."echo "Running PR-Agent commands..."

or inserts additional, unexpected lines into the diff.

---

Expected Behavior

• Each code suggestion should reflect the correct diff format.
• Multi-line suggestions should maintain proper line breaks.
• Each suggestion block should contain only the intended changed lines.
• No concatenation or duplication of strings inside green/red diff blocks.
• Suggestions must be in a format that Azure DevOps can actually apply.

---

Actual Behavior

• Suggested code lines are concatenated together without spacing or newlines.
• Red/green diff lines include merged tokens from adjacent lines.
• Review suggestions combine unrelated edits into one block.
• The resulting suggestion cannot be applied or copied manually.
• Happens with multiple file types: .sh, .yml, .toml.

This occurs consistently even after significant .pr_agent.toml configuration tuning.

---

Attempted Configuration Fixes (none resolved the bug)

I tuned many settings to reduce noise, limit suggestions, and ensure formatting correctness:

• committable_code_suggestions = true
• num_code_suggestions_per_chunk = 2 or 4
• focus_only_on_problems = true
• suggestions_score_threshold = 40–75
• persistent_comment = true
• publish_output_no_suggestions = false
• Extensive extra_instructions to avoid trivial suggestions
• Added ignore rules:

ignore.regex = [
  ".*\\.pr_agent\\.toml$",
  "azure-pipelines-pr-agent\\.yml$"
]

However, none of these resolved the core formatting bug.

---

Relevant Pipeline Execution Context

Azure DevOps pipeline job:

container:
  image: codiumai/pr-agent:latest
  options: --entrypoint ""

Command execution:

pr-agent --pr_url="$PR_URL" describe
pr-agent --pr_url="$PR_URL" review
pr-agent --pr_url="$PR_URL" improve --pr_code_suggestions.committable_code_suggestions=true

.pr_agent.toml contains valid configuration (included fully in the issue if needed).

---

Reproduction Steps

1. Use Azure DevOps PR with PR-Agent running inside DevOps pipeline.
2. Include scripts or config files .yaml, .toml, .sh with multi-line structures.
3. Trigger PR-Agent (review, describe, improve).
4. Open PR → View inline code suggestions.

Result: suggestion blocks appear malformed, merged, duplicated, or incorrectly colored.

---

Environment

• Qodo PR-Agent version: latest Docker image (codiumai/pr-agent:latest)
• Platform: Azure DevOps
• Pipeline: YAML-based Linux container job
• Runner: SHARED-BUILD-NIX
• Shell: bash

---

Additional Notes

This appears to be a formatting/line-break encoding issue inside the diff suggestion generator, specifically in how suggestions are serialized for Azure DevOps API.

It does not seem configuration-related.

Screenshots:

whole files:
.pr_agent.toml

[pr_code_suggestions]
# Enable inline code suggestions for Azure DevOps
committable_code_suggestions = true

# Number of code suggestions per code chunk - reduced to show only best suggestions
num_code_suggestions_per_chunk = 2

# Focus on actual problems rather than just style improvements
focus_only_on_problems = true

# High threshold to filter out trivial suggestions (0-100, higher = fewer but better suggestions)
suggestions_score_threshold = 75

# Keep suggestions visible in PR comments
persistent_comment = true

# Don't show output if no suggestions found
publish_output_no_suggestions = false

# Enable chat interaction with QM bot on suggestions
enable_chat_in_code_suggestions = true

# Extra instructions to filter out low-quality suggestions
extra_instructions = """
IMPORTANT: Only suggest changes that have significant impact on:
- Security vulnerabilities
- Performance issues
- Logic errors or bugs
- Code correctness

DO NOT suggest:
- Minor formatting or style changes
- Trivial typo fixes in comments or variable names
- Configuration value changes without clear justification
"""

[pr_reviewer]
# Maximum number of review findings to display - reduced to show only critical issues
num_max_findings = 5

# Enable comprehensive review sections
require_tests_review = true
require_security_review = true
require_score_review = false

# Keep review comments persistent
persistent_comment = true

# Don't publish if no meaningful findings
publish_output_no_suggestions = false

# Extra instructions for high-quality reviews only
extra_instructions = """
Focus on critical issues only:
- Security vulnerabilities
- Potential runtime errors
- Data integrity issues
- Performance bottlenecks
- Logic errors that could cause incorrect behavior

Ignore trivial issues like:
- Code style preferences
- Minor formatting inconsistencies
- Trivial naming suggestions
"""

[azure_devops]
# Set comment status to active for visibility
default_comment_status = "active"

[config]
# Set git provider
git_provider = "azure"

azure-pipelines-pr-agent.yml:

pr:
  branches:
    include:
      - "*"

stages:
  - stage: QodoMerge
    displayName: "Qodo Merge - AI Code Review"
    condition: eq(variables['Build.Reason'], 'PullRequest')
    jobs:
      - job: QodoMergeJob
        displayName: "Run Qodo Merge"
        pool: 
          name: SHARED-BUILD-NIX
        container:
          image: codiumai/pr-agent:latest
          options: --entrypoint ""
        variables:
          - group: QODO-TEST-HONZA
        steps:
          - script: |
              echo "Build Reason: $(Build.Reason)"
              echo "Is PR: ${{ eq(variables['Build.Reason'], 'PullRequest') }}"
            displayName: "Debug Build Info"

          - script: |
              set -euo pipefail
              echo "Running Qodo Merge on PR"

              if [ -z "${SYSTEM_PULLREQUEST_PULLREQUESTID:-}" ]; then
                echo "Error: SYSTEM_PULLREQUEST_PULLREQUESTID is not set. This step should run only for PR validations."
                exit 1
              fi

              required_vars=("SYSTEM_COLLECTIONURI" "SYSTEM_TEAMPROJECT" "BUILD_REPOSITORY_NAME" "SYSTEM_PULLREQUEST_PULLREQUESTID")
              for v in "${required_vars[@]}"; do
                if [ -z "${!v:-}" ]; then
                  echo "Error: required variable '$v' is not set."
                  exit 1
                fi
              done

              ORG_URL="${SYSTEM_COLLECTIONURI%/}" # Remove trailing slash if present

              PR_URL="${ORG_URL}/${SYSTEM_TEAMPROJECT}/_git/${BUILD_REPOSITORY_NAME}/pullrequest/${SYSTEM_PULLREQUEST_PULLREQUESTID}"

              export azure_devops__org="$ORG_URL"
              export config__git_provider="azure"
              export azure_devops__default_comment_status="active"

              # Verify configuration file exists
              if [ -f .pr_agent.toml ]; then
                echo "Found .pr_agent.toml configuration file"
              else
                echo "WARNING: .pr_agent.toml not found"
              fi

              echo "Running PR-Agent commands..."
              pr-agent --pr_url="$PR_URL" describe
              pr-agent --pr_url="$PR_URL" review
              pr-agent --pr_url="$PR_URL" improve --pr_code_suggestions.committable_code_suggestions=true
            env:
              azure_devops__pat: $(azure_devops_pat)
              openai__key: $(OPENAI_KEY)
            displayName: "Run Qodo Merge Commands"

[qodo-merge-for-open-source]

💡 Auto-generated documentation-based answer:

This appears to be a known formatting issue with inline code suggestions in Azure DevOps. The problem you're experiencing—where suggestions are concatenated, duplicated, or malformed—is related to how PR-Agent formats diff suggestions for the Azure DevOps API.

Key Points

1. Platform Limitation: Azure DevOps has different API capabilities compared to GitHub/GitLab. The inline code suggestion formatting may not be fully compatible with Azure DevOps's comment system.

2. Table vs Committable Format: According to the documentation, PR-Agent supports two modes for presenting code suggestions:

   • Table mode (default and recommended): Presents suggestions in a cleaner, more organized table format
   • Inline Committable mode: Shows suggestions as inline code comments

3. Azure DevOps Specific Behavior: The documentation notes that "Bitbucket cloud and server supports only committable code comments mode" due to platform limitations. Azure DevOps may have similar constraints.

Recommended Solutions

Option 1: Use Table Mode (Recommended)

Try disabling committable code suggestions to use the default table format:

[pr_code_suggestions]
commitable_code_suggestions = false

Or via command line:

pr-agent --pr_url="$PR_URL" improve --pr_code_suggestions.commitable_code_suggestions=false

The table format offers several advantages:

• Reduced noise and cleaner PR experience
• Better organization with one-liner summaries
• Less prone to formatting issues
• Easier to review and prioritize

Option 2: Report as Bug

Since you've tried extensive configuration changes without success, this appears to be a platform-specific bug in how PR-Agent serializes suggestions for Azure DevOps. Consider:

1. Opening a GitHub issue in the PR-Agent repository
2. Including:
   • Raw JSON output from PR-Agent
   • Screenshots of malformed suggestions
   • Your .pr_agent.toml configuration
   • Example PR links (if possible)

Option 3: Alternative Workflow

If inline suggestions are critical for your workflow, consider:

• Using the review tool instead of improve for feedback
• Manually applying suggestions based on the table format
• Waiting for a fix if this is confirmed as a bug

Additional Context Needed

To provide more specific guidance, it would help to know:

• Are you using Azure DevOps Cloud or Server?
• What does the raw API response look like from PR-Agent?
• Does the table format work correctly in your environment?

Related Configuration

You might also want to review these settings:

[pr_code_suggestions]
persistent_comment = true
dual_publishing_score_threshold = -1  # Disable dual publishing

Relevant Sources:

• https://qodo-merge-docs.qodo.ai//tools/improve#table-vs-committable-code-comments
• https://qodo-merge-docs.qodo.ai//installation/azure#azure-devops-pipeline
• https://qodo-merge-docs.qodo.ai//tools/improve#configuration-options
• https://qodo-merge-docs.qodo.ai//usage-guide/automations_and_usage#azure-devops-provider