Skip to content

Enforce OpenAI generation penalty semantics - #1076

Open
Baiju Meswani (baijumeswani) wants to merge 2 commits into
mainfrom
baijumeswani/penalties
Open

Enforce OpenAI generation penalty semantics#1076
Baiju Meswani (baijumeswani) wants to merge 2 commits into
mainfrom
baijumeswani/penalties

Conversation

@baijumeswani

Copy link
Copy Markdown
Collaborator

Summary

Foundry Local currently maps OpenAI frequency_penalty and presence_penalty values to ONNX Runtime GenAI options with different meanings.

This can produce incorrect output. GitHub Copilot CLI also sends both fields with the neutral value 0.

This change treats zero penalties as no-ops and rejects nonzero penalties before inference.

Behavior

  • frequency_penalty: 0 is accepted and does not change model settings.
  • presence_penalty: 0 is accepted and does not change model settings.
  • Nonzero penalty values return a client error.
  • Foundry Local does not map frequency penalty to repetition penalty.
  • Foundry Local does not map presence penalty to diversity penalty.
  • Chat Completions and Responses use the same behavior.
  • Direct SDK requests use the same behavior.

Why

OpenAI frequency and presence penalties are additive token-frequency controls.

ONNX Runtime GenAI repetition and diversity penalties have different semantics.

Silently mapping between them is not correct. Rejecting unsupported values is safer and makes the limitation clear.

Testing

  • Added tests that zero penalties do not override model defaults.
  • Added tests that positive and negative nonzero penalties are rejected.
  • Added Chat Completions and Responses converter coverage.
  • Added endpoint coverage that verifies rejection happens before inference.
  • Updated Python serialization tests.
  • Updated C, C++, Python, C#, and Rust documentation.

Copilot AI balanced review requested due to automatic review settings September 6, 2026 16:21
@vercel

vercel Bot commented Sep 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
foundry-local Ready Ready Preview Sep 6, 2026 6:22pm UTC

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Public C, C#, and Rust API documentation remains incomplete or contradictory about the zero-only constraint.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Enforces OpenAI penalty semantics by accepting zero as a no-op and rejecting nonzero values before inference.

Changes:

  • Removes incorrect ORT penalty mappings.
  • Adds validation across endpoints, converters, and direct SDK requests.
  • Updates tests and multi-language documentation.
File summaries
File Description
sdk_v2/rust/README.md Documents zero-only penalties.
sdk_v2/rust/docs/api.md Updates builder API guidance.
sdk_v2/python/test/unit/test_session_types.py Tests zero serialization.
sdk_v2/python/test/unit/test_chat_settings.py Updates settings serialization coverage.
sdk_v2/python/README.md Documents penalty restriction.
sdk_v2/cs/README.md Updates C# example.
sdk_v2/cpp/test/sdk_api/responses_test.cc Tests Responses rejection.
sdk_v2/cpp/test/sdk_api/reasoning_model_test.cc Removes unsupported penalties.
sdk_v2/cpp/test/sdk_api/chat_completions_test.cc Tests Chat Completions rejection.
sdk_v2/cpp/test/internal_api/response_converter_test.cc Covers Responses conversion semantics.
sdk_v2/cpp/test/internal_api/chat/search_options_test.cc Tests direct SDK behavior.
sdk_v2/cpp/test/internal_api/chat_completions_converter_test.cc Covers chat conversion semantics.
sdk_v2/cpp/src/service/responses_handler.cc Rejects unsupported penalties early.
sdk_v2/cpp/src/service/chat_completions_handler.cc Adds endpoint validation.
sdk_v2/cpp/src/inferencing/generative/openresponses/response_converter.cc Stops forwarding penalties.
sdk_v2/cpp/src/inferencing/generative/chat/search_options.h Documents supported values.
sdk_v2/cpp/src/inferencing/generative/chat/search_options.cc Removes incorrect ORT mappings.
sdk_v2/cpp/src/contracts/responses.h Documents Responses contracts.
sdk_v2/cpp/src/contracts/chat_completions.h Documents chat contracts.
sdk_v2/cpp/src/contracts/chat_completions_converter.cc Validates chat parameters.
sdk_v2/cpp/include/foundry_local/foundry_local_cpp.h Updates C++ API documentation.
sdk_v2/cpp/include/foundry_local/foundry_local_c.h Updates C parameter documentation.
Review details
  • Files reviewed: 22/22 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk_v2/cpp/include/foundry_local/foundry_local_c.h Outdated
Comment thread sdk_v2/cs/README.md
Comment thread sdk_v2/rust/docs/api.md
Document the zero-only penalty contract in the C macros, C# IntelliSense, and Rust source rustdocs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bb683a10-24f0-42bd-85fc-8babb4878bd8

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The behavior is consistently enforced across API layers with focused coverage for zero, positive, and negative values.

Review details
  • Files reviewed: 26/26 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants