fix colab notebooks#20
Open
tburleyinfo wants to merge 5 commits into
Open
Conversation
Signed-off-by: Timothy Burley <34224160+tburleyinfo@users.noreply.github.com>
97ff92a to
4547d44
Compare
…ents. Signed-off-by: Timothy Burley <34224160+tburleyinfo@users.noreply.github.com>
Signed-off-by: Timothy Burley <34224160+tburleyinfo@users.noreply.github.com>
Signed-off-by: Timothy Burley <34224160+tburleyinfo@users.noreply.github.com>
4547d44 to
b833582
Compare
Signed-off-by: Timothy Burley <34224160+tburleyinfo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Type of contribution
Hit cuda13 compatibility and dependency errors in colab environment. Resolved by uninstalling cuda13 and related dependencies and using cuda128 and related dependencies. Requires the install cell to be ran twice to have the new packages in the runtime environment. Run the install cell once, then restart the runtime (do not disconnect and delete) and then run all cells.
Files modified
notebooks/*colab.ipynb
The following files are core infrastructure. Please discuss with maintainers before modifying:
hook_llm.py,_hook_plugin.py,registry.py,hook_client.py,run_utils.py,shm_utils.py,workers/_common.py__init__.pyPlugin architecture checklist
PluginRegistryin__init__.pyworker_extension_cls(for in-band artifact retrieval and concurrent request support)Testing
Ran each notebook to ensure each would complete.
Related issue
Contribution acknowledgement
If this contribution is included in a future version of the vLLM-Hook technical report, would you like to be credited as a co-author?
If yes, please provide: