Fix: Respect reasoning_effort config for GPT-5 models

[Tyler-Rak]

User description

Fixes #2120

Problem

GPT-5 models were hardcoded to use reasoning_effort='minimal', ignoring the user's config.reasoning_effort setting.

Solution

Modified litellm_ai_handler.py to:

• Read reasoning_effort from configuration instead of hardcoding
• Support values: 'none', 'low', 'medium', 'high'
• Default to 'none' for non-thinking models, 'low' for thinking models
• Add logging to show which reasoning effort level is being used

Testing

Verified with GPT-5 model that:

• Config setting reasoning_effort = "medium" is now respected
• Logs confirm: "Using reasoning_effort=medium for GPT-5 model (from config)"
• Model correctly uses medium reasoning effort in API calls

---

PR Type

Bug fix

---

Description

• Respect user's reasoning_effort config setting for GPT-5 models

• Replace hardcoded 'minimal'/'low' values with configurable defaults

• Support reasoning effort values: 'none', 'low', 'medium', 'high'

• Add logging to show which reasoning effort level is being used

---

Diagram Walkthrough

flowchart LR
  A["GPT-5 Model Request"] --> B{"Model Type?"}
  B -->|"Thinking Model"| C["Use config or default 'low'"]
  B -->|"Non-thinking Model"| D["Use config or default 'none'"]
  C --> E["Set reasoning_effort parameter"]
  D --> E
  E --> F["Log effort level used"]

Loading

File Walkthrough

	Relevant files
Bug fix	litellm_ai_handler.py Make GPT-5 reasoning_effort configurable with smart defaults pr_agent/algo/ai_handlers/litellm_ai_handler.py Read reasoning_effort from config instead of hardcoding values Use 'low' as default for thinking models, 'none' for non-thinking models Validate config value against supported efforts list Add info logging to display which reasoning effort is being used +17/-8

---

[qodo-free-for-open-source-projects]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🟡	🎫 #2120 🟢 Fix the root cause in litellm_ai_handler.py where reasoning_effort is hardcoded Respect the user's reasoning_effort configuration setting instead of hardcoding values Allow users to configure reasoning_effort in .pr_agent.toml and have it properly applied 🔴 Fix the issue where gpt-5.1 and gpt-5.1-codex always receive reasoning_effort='minimal', causing API failures Support the correct reasoning_effort values for gpt-5.1-codex ('low', 'medium', 'high') and avoid using 'minimal' ⚪ Verify that the fix works correctly with actual OpenAI API calls for gpt-5.1-codex model Confirm that the default 'none' value works for gpt-5.1 non-thinking models Test that gpt-5.1-codex properly accepts 'low', 'medium', and 'high' values from config
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Consistent Naming Conventions Objective: All new variables, functions, and classes must follow the project's established naming standards Status: Passed
	No Dead or Commented-Out Code Objective: Keep the codebase clean by ensuring all submitted code is active and necessary Status: Passed
	Robust Error Handling Objective: Ensure potential errors and edge cases are anticipated and handled gracefully throughout the code Status: Passed
	Single Responsibility for Functions Objective: Each function should have a single, well-defined responsibility Status: Passed
	When relevant, utilize early return Objective: In a code snippet containing multiple logic conditions (such as 'if-else'), prefer an early return on edge cases than deep nesting Status: Passed
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-free-for-open-source-projects]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	✅ Refactor logic to improve readability Suggestion Impact: The commit implements the exact refactoring suggested. It stores the validation result in `is_config_valid` variable, adds a `source` variable to track whether config or default is used, restructures the conditional logic to eliminate redundant checks, adds a warning log for invalid configuration values, and updates the info log to use the `source` variable with quoted effort value. code diff: - - if model.endswith('_thinking'): - # For thinking models, use config value or default to 'low' - effort = config_effort if config_effort in supported_efforts else 'low' + is_config_valid = config_effort in supported_efforts + source = "config" + + if is_config_valid: + effort = config_effort else: - # For non-thinking models, use config value or default to 'none' - # If 'none' fails for specific models (e.g., codex), they should set config to 'low' - effort = config_effort if config_effort in supported_efforts else 'none' + source = "default" + if config_effort is not None: + get_logger().warning( + f"Invalid reasoning_effort '{config_effort}' in config. " + f"Using default. Supported values: {supported_efforts}" + ) + if model.endswith('_thinking'): + effort = 'low' + else: + effort = 'none' thinking_kwargs_gpt5 = { "reasoning_effort": effort, "allowed_openai_params": ["reasoning_effort"], } - get_logger().info(f"Using reasoning_effort={effort} for GPT-5 model (from {'config' if config_effort in supported_efforts else 'default'})") + get_logger().info(f"Using reasoning_effort='{effort}' for GPT-5 model (from {source})") Refactor the reasoning_effort logic to remove redundant checks by storing the validation result in a variable, and add a warning log for invalid configuration values. pr_agent/algo/ai_handlers/litellm_ai_handler.py [295-313] # Respect user's reasoning_effort config setting # Supported values: 'none', 'low', 'medium', 'high' # Note: gpt-5.1 supports 'none', but gpt-5.1-codex does not config_effort = get_settings().config.reasoning_effort supported_efforts = ['none', 'low', 'medium', 'high'] +is_config_valid = config_effort in supported_efforts +source = "config" -if model.endswith('_thinking'): - # For thinking models, use config value or default to 'low' - effort = config_effort if config_effort in supported_efforts else 'low' +if is_config_valid: + effort = config_effort else: - # For non-thinking models, use config value or default to 'none' - # If 'none' fails for specific models (e.g., codex), they should set config to 'low' - effort = config_effort if config_effort in supported_efforts else 'none' + source = "default" + if config_effort is not None: + get_logger().warning( + f"Invalid reasoning_effort '{config_effort}' in config. " + f"Using default. Supported values: {supported_efforts}" + ) + if model.endswith('_thinking'): + effort = 'low' + else: + effort = 'none' thinking_kwargs_gpt5 = { "reasoning_effort": effort, "allowed_openai_params": ["reasoning_effort"], } -get_logger().info(f"Using reasoning_effort={effort} for GPT-5 model (from {'config' if config_effort in supported_efforts else 'default'})") +get_logger().info(f"Using reasoning_effort='{effort}' for GPT-5 model (from {source})") [Suggestion processed] Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies redundant checks and proposes a refactoring that improves readability. It also enhances usability by adding a warning log for invalid configuration values, which is a valuable improvement.	Low
Learned best practice	Use safe configuration attribute access Use safe attribute access with getattr() to handle cases where config.reasoning_effort may not exist. This prevents AttributeError if the configuration attribute is missing or None. pr_agent/algo/ai_handlers/litellm_ai_handler.py [298-307] -config_effort = get_settings().config.reasoning_effort +config = getattr(get_settings(), 'config', None) +config_effort = getattr(config, 'reasoning_effort', None) if config else None supported_efforts = ['none', 'low', 'medium', 'high'] if model.endswith('_thinking'): # For thinking models, use config value or default to 'low' effort = config_effort if config_effort in supported_efforts else 'low' else: # For non-thinking models, use config value or default to 'none' # If 'none' fails for specific models (e.g., codex), they should set config to 'low' effort = config_effort if config_effort in supported_efforts else 'none' Apply / Chat Suggestion importance[1-10]: 6 __ Why: Relevant best practice - Use safe dictionary access methods like .get() with default values when accessing configuration attributes that may not exist to prevent AttributeError exceptions at runtime.	Low
Update

• Author self-review: I have reviewed the PR code suggestions, and addressed the relevant ones.

[Tyler-Rak]

Code review suggestions addressed:

✅ Suggestion 1 (Refactor logic): Implemented in commit 956f366

• Improved readability by storing validation result in variable
• Added warning log for invalid configuration values

❌ Suggestion 2 (getattr defensive coding): Not implementing

• get_settings() always returns a valid Dynaconf object
• config.reasoning_effort is always defined with default value "medium" in configuration.toml
• Existing validation already handles None/invalid values
• Adding getattr() would add unnecessary complexity

All relevant suggestions have been addressed.

[naorpeled]

Hey @Tyler-Rak,
first of all, this looks great, thanks for your contribution!

Can you please add tests to validate this?

[Tyler-Rak]

Hi @naorpeled , sure I have added the UT for this.

[shun-tak]

GPT-5 supports reasoning_effort = minimal but not none. I suspect this code will throw an error on this line when passing ‘gpt-5’ as the model.

Ref: https://platform.openai.com/chat/edit?models=gpt-5-2025-08-07

[shun-tak]

Thanks for the PR — and sorry if my previous comment sounded blunt. I meant it purely as a technical point. Appreciate the effort here.

[Tyler-Rak]

@shun-tak
Thanks for raising this up.

Yes for GPT 5 this is the case, but OpenAI changed this for later models, and here is the list:

Model	Default	Valid Values	Notes
GPT-5	medium	minimal, low, medium, high	Does NOT support none
GPT-5.1	none	none, low, medium, high	Does NOT support minimal
GPT-5.2	none	none, minimal, low, medium, high, xhigh	Supports both none and minimal + new xhigh

I chose to use the latest setup, but yes this is risky for anyone using the old models.

How about this: when user is not setting the reasoning parameters, instead pass a default value, we ignore that parameters and let openAI decide the parameter?

[shun-tak]

Thank you for your comment. Setting OpenAI's default values may lead to unintended results for PR Agent users because the default values differ depending on the model.

• gpt-5 => medium
• gpt-5.1 => none
• gpt-5.2 => none

Looking at Codex, they set reasoning effort to medium by default because it strikes a good balance between speed and accuracy. For PR Agent as well, I feel using medium is generally preferable.

Also, I'm a bit concerned about model.endswith(‘_thinking’). This means users have to configure something like ‘gpt-5_thinking’. Is that correct? I feel like users aren't aware of this feature. If it doesn't seem to be widely used, we might consider removing it in a future version. Doing so would simplify the logic around the default value.

[Tyler-Rak]

ah, so you recommending to simply set to medium for all case?

[shun-tak]

Yes!

[Tyler-Rak]

ah ok, that make sense, I have updated accordingly, by

1. Simplified the default logic(I removed the thinking branch since you mentioned and I also think it's kind of concerning, but if you want to keep it for now just let me know).
2. Added the new reasoning effort type into the Enum. Since I'm actually using the newer versions of GPT.
3. Slightly changed the validation mechanisms.

Please let me know if the new code looks good to you.

[shun-tak]

Thank you for the change! Looks good!

[Tyler-Rak]

Thanks @shun-tak .

Sorry this is my first contribute and I noticed that I cannot merge the PR.

May I ask will any of the owner of this repo merge this later? Is there anything else needed from my side?

[naorpeled]

I'll review the new changes soon and see if there's anything to adjust, if there won't be any changes to be done I'll merge the PR