Skip to content

Improvement Request: Add parameter to support decoding_method="sample" #248

Open
@hirokik1

Description

@hirokik1

Hello,

I have been using the watsonx-openapi.json file for creating an extension for the watsonx Assistant, and I would like to express my gratitude for making it available.
integrations/extensions/starter-kits/language-model-watsonx/watsonx-openapi.json

I am writing to submit a request for an improvement.
I would appreciate it if you could add the following parameters to the parameters section, so that it becomes possible to execute the API with the decoding_method set to "sample".
The description below is quoted from the BAM explanation site. I have tested this in my environment and confirmed it works.

"decoding_method": {
  "type": "string",
  "description": "Represents the strategy used for picking the tokens during generation of the output text. Options are greedy and sample. Value defaults to sample if not specified.",
  "example": "greedy"
},
"top_k": {
  "type": "number",
  "description": "The number of highest probability vocabulary tokens to keep for top-k-filtering. Only applies for sampling mode, with range from 1 to 100. When decoding_strategy is set to sample, only the top_k most likely tokens are considered as candidates for the next generated token.",
  "example": "50"
},
"top_p": {
  "type": "number",
  "description": "Similar to top_k except the candidates to generate the next token are the most likely tokens with probabilities that add up to at least top_p. The valid range is 0.0 to 1.0 where 1.0 is equivalent to disabled and is the default. Also known as nucleus sampling.",
  "example": "1"
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions