Conversation
top_k, function_to_apply, and batch_size are call-time parameters passed at pipe() invocation, not pipeline construction config. Including them in the cache key created duplicate pipeline instances in memory for each unique parameter combination. Now a single pipeline is reused regardless of top_k/function_to_apply values — these are passed per-call at inference time. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the Hugging Face audio classification pipeline caching mechanism to use only model identity and device as keys, treating parameters like top_k and function_to_apply as call-time arguments. Feedback indicates that these parameters, along with batch_size, should be removed from the _get_hf_audio_classification_pipeline function signature to avoid confusion, as they are no longer utilized within that scope. Additionally, it was noted that batch_size is currently ignored during the pipeline invocation.
| # Cache by model identity + device only. top_k, function_to_apply, | ||
| # and batch_size are call-time parameters passed at pipe() invocation. |
There was a problem hiding this comment.
The comment states that batch_size is a call-time parameter passed at the pipe() invocation, but it is not actually passed to the pipeline call on line 149. This means the batch_size parameter (which defaults to 16) is currently ignored, and the pipeline will use its default batch size (typically 1).
Additionally, since top_k, function_to_apply, and batch_size are no longer used within _get_hf_audio_classification_pipeline, they should be removed from this function's signature and the call site at line 108 to improve maintainability and avoid confusion.
1 participant
Cache by model+device only. Call-time params not in key.