[Module] Add vLLM module application#943
Merged
Eyal-Danieli merged 11 commits intomlrun:developmentfrom Dec 24, 2025
Merged
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
guylei-code
requested review from
Eyal-Danieli,
GiladShapira94 and
danielperezz
Correct the format of the generationDate field.
Added copyright information and module description.
Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com>
Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com>
Updated the description to provide more details about the vLLM module's functionality and configuration options.
Added docstring to VLLMModule class to describe its purpose and functionality.
Eyal-Danieli
approved these changes
Dec 24, 2025
Eyal-Danieli
added a commit
to Eyal-Danieli/functions
that referenced
this pull request
Dec 28, 2025
* vLLM module first commit * Chane the module name * Fix generationDate format in item.yaml Correct the format of the generationDate field. * Fix test assertion for vllm_proxy_app type * Add copyright and module description to vllm_module.py Added copyright information and module description. * Update modules/src/vllm_module/vllm_module.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * Update modules/src/vllm_module/vllm-module.ipynb Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * Enhance description of vLLM module Updated the description to provide more details about the vLLM module's functionality and configuration options. * Add docstring for VLLMModule class Added docstring to VLLMModule class to describe its purpose and functionality. * Add validation for tensor_parallel_size against gpus * Update item.yaml --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com>
This was referenced Dec 28, 2025
Eyal-Danieli
added a commit
that referenced
this pull request
Dec 28, 2025
* replace author to Iguazio manually (#905) * Organize CLI directory + new CLI for generating item.yaml files (#906) * create a CLI for generating item.yaml and organize the CLI directory * modify comments to module * PR fixes * Update cli/common/generate_item_yaml.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * fill count events notebook (#908) * avoid noise reduction unit test (#909) * Add histogram-data-drift monitoring application module (without example) (#911) * histogram data drift module with empty example notebook * post review fixes * chore(readme): auto-update asset tables [skip ci] * Fill histogram-data-drift example notebook (#912) * fill data-drift nb * post review fixes * Add evidently demo app monitoring application module (without example) (#913) * sphinx build docs bug fix * add evidently demo app module (empty example notebook) * post review changes * chore(readme): auto-update asset tables [skip ci] * [Translate] Require torch>=2.6 for the translate function to work properly (#915) * lock torch valid version * edit the item.yaml and generated function.yaml * update mlrun version * [CLI] Generated READMEs are produced with broken links to the items (#918) * fix * test fix * test fix * test fix * test fix * final workflow * chore(readme): auto-update asset tables [skip ci] * OpenAI Module without notebook (#917) * First commit OpenAI Module * First commit OpenAI Module * Update example filename in item.yaml * Delete modules/src/openai_proxy/requirements.txt No need due to no unitest * Update item.yaml for OpenAI application configuration * Update modules/src/openai_proxy/openai.py Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * Change category name from 'GenAI' to 'genai' * Update package requirements with version constraints * Second commit adding notebook * Refactor OpenAI proxy to use base64 encoded script Refactor OpenAI proxy implementation to use base64 encoded script and update FastAPI app configuration. * Change deployment method to OpenAIModule * Third commit adding notebook * Third commit adding notebook * Remove package requirements from item.yaml Removed specific requirements for fastapi and requests. * Rename item and update kind in YAML * Update openai.py * Third commit adding notebook * Fix after review * Fix after review --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * [Evidently] Fill example notebook (#919) * add notebook + rename directory + correct evidently version * remove extra cell * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * [CLI + Modules] Fix time format in generate item yaml script (#922) * fix time format for evidently and hist * fix cli script * fix datetime format * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * Fix CMD first commit * Fix CMD second commit * remove max-width restriction from the main content (#929) * add test, requirement file and notebook * fix cli/utils/helpers.py * [Modules] Modify Evidently & Histogram monitoring apps example notebooks to the change in evaluate() (#934) * histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently dependency * add dependency * remove [ui] from evidently dependency * change notebook name to: openai_proxy_app * [Docs] Add guidelines for contributing new functions or modules (#931) * CONTRIBUTING.md * CONTRIBUTING.md * improvements * [Module] Updated the requirements for OpenAI module (#939) * Updated the requirements * Added Deafult model for credentials * Pin dependencies to specific versions in item.yaml * Update dependencies in openai_proxy_app.py * Specify versions for dependencies in requirements.txt * Fix environment variable access for OPENAI_DEFAULT_MODEL * change notebook name to: openai_proxy_app * [Module] agent_deployer module (#937) * first commit * added all modules * fixes from review and notebook adjustments * added unit tests and documentation * remove log * small fix * cr fixes * get_function fix * cr fixes working deployment * small fix notebook * chore(readme): auto-update asset tables [skip ci] * [Infra] Add 'Steps' to 'mlrun/hub-assets' and adjust CI/CD workflow to support it (#942) * first draft to support steps * correct branch name to test workflow * mlrun version * fix test * create a marketplace artifact * final workflow * add docstring to test the ui docs html * modify the readme to different colums set * no need for now * remove the dummy step files for now * chore(readme): auto-update asset tables [skip ci] * add default-handler to the step template (#947) * [AgentDeployer] Notebook prompt fix (#946) * Cherry Pick from Development (#941) * replace author to Iguazio manually (#905) * Organize CLI directory + new CLI for generating item.yaml files (#906) * create a CLI for generating item.yaml and organize the CLI directory * modify comments to module * PR fixes * Update cli/common/generate_item_yaml.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * fill count events notebook (#908) * avoid noise reduction unit test (#909) * Add histogram-data-drift monitoring application module (without example) (#911) * histogram data drift module with empty example notebook * post review fixes * chore(readme): auto-update asset tables [skip ci] * Fill histogram-data-drift example notebook (#912) * fill data-drift nb * post review fixes * Add evidently demo app monitoring application module (without example) (#913) * sphinx build docs bug fix * add evidently demo app module (empty example notebook) * post review changes * chore(readme): auto-update asset tables [skip ci] * [Translate] Require torch>=2.6 for the translate function to work properly (#915) * lock torch valid version * edit the item.yaml and generated function.yaml * update mlrun version * [CLI] Generated READMEs are produced with broken links to the items (#918) * fix * test fix * test fix * test fix * test fix * final workflow * chore(readme): auto-update asset tables [skip ci] * OpenAI Module without notebook (#917) * First commit OpenAI Module * First commit OpenAI Module * Update example filename in item.yaml * Delete modules/src/openai_proxy/requirements.txt No need due to no unitest * Update item.yaml for OpenAI application configuration * Update modules/src/openai_proxy/openai.py Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * Change category name from 'GenAI' to 'genai' * Update package requirements with version constraints * Second commit adding notebook * Refactor OpenAI proxy to use base64 encoded script Refactor OpenAI proxy implementation to use base64 encoded script and update FastAPI app configuration. * Change deployment method to OpenAIModule * Third commit adding notebook * Third commit adding notebook * Remove package requirements from item.yaml Removed specific requirements for fastapi and requests. * Rename item and update kind in YAML * Update openai.py * Third commit adding notebook * Fix after review * Fix after review --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * [Evidently] Fill example notebook (#919) * add notebook + rename directory + correct evidently version * remove extra cell * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * [CLI + Modules] Fix time format in generate item yaml script (#922) * fix time format for evidently and hist * fix cli script * fix datetime format * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * Fix CMD first commit * Fix CMD second commit * remove max-width restriction from the main content (#929) * add test, requirement file and notebook * fix cli/utils/helpers.py * [Modules] Modify Evidently & Histogram monitoring apps example notebooks to the change in evaluate() (#934) * histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently dependency * add dependency * remove [ui] from evidently dependency * change notebook name to: openai_proxy_app * [Docs] Add guidelines for contributing new functions or modules (#931) * CONTRIBUTING.md * CONTRIBUTING.md * improvements * [Module] Updated the requirements for OpenAI module (#939) * Updated the requirements * Added Deafult model for credentials * Pin dependencies to specific versions in item.yaml * Update dependencies in openai_proxy_app.py * Specify versions for dependencies in requirements.txt * Fix environment variable access for OPENAI_DEFAULT_MODEL * change notebook name to: openai_proxy_app * [Module] agent_deployer module (#937) * first commit * added all modules * fixes from review and notebook adjustments * added unit tests and documentation * remove log * small fix * cr fixes * get_function fix * cr fixes working deployment * small fix notebook * chore(readme): auto-update asset tables [skip ci] --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: guylei-code <guyleibu@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com> Co-authored-by: Roy Schossberger <85231212+royischoss@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * small fix notebook * remove changes in README.md * remove changes in README.md --------- Co-authored-by: Eyal Danieli <eyald@iguazio.com> Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: guylei-code <guyleibu@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com> * [Module] Add vLLM module application (#943) * vLLM module first commit * Chane the module name * Fix generationDate format in item.yaml Correct the format of the generationDate field. * Fix test assertion for vllm_proxy_app type * Add copyright and module description to vllm_module.py Added copyright information and module description. * Update modules/src/vllm_module/vllm_module.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * Update modules/src/vllm_module/vllm-module.ipynb Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * Enhance description of vLLM module Updated the description to provide more details about the vLLM module's functionality and configuration options. * Add docstring for VLLMModule class Added docstring to VLLMModule class to describe its purpose and functionality. * Add validation for tensor_parallel_size against gpus * Update item.yaml --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * chore(readme): auto-update asset tables [skip ci] * fix nb name (#948) * [Module] vLLM module with notebook (#952) * Cherry Pick from Development (#941) * replace author to Iguazio manually (#905) * Organize CLI directory + new CLI for generating item.yaml files (#906) * create a CLI for generating item.yaml and organize the CLI directory * modify comments to module * PR fixes * Update cli/common/generate_item_yaml.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * fill count events notebook (#908) * avoid noise reduction unit test (#909) * Add histogram-data-drift monitoring application module (without example) (#911) * histogram data drift module with empty example notebook * post review fixes * chore(readme): auto-update asset tables [skip ci] * Fill histogram-data-drift example notebook (#912) * fill data-drift nb * post review fixes * Add evidently demo app monitoring application module (without example) (#913) * sphinx build docs bug fix * add evidently demo app module (empty example notebook) * post review changes * chore(readme): auto-update asset tables [skip ci] * [Translate] Require torch>=2.6 for the translate function to work properly (#915) * lock torch valid version * edit the item.yaml and generated function.yaml * update mlrun version * [CLI] Generated READMEs are produced with broken links to the items (#918) * fix * test fix * test fix * test fix * test fix * final workflow * chore(readme): auto-update asset tables [skip ci] * OpenAI Module without notebook (#917) * First commit OpenAI Module * First commit OpenAI Module * Update example filename in item.yaml * Delete modules/src/openai_proxy/requirements.txt No need due to no unitest * Update item.yaml for OpenAI application configuration * Update modules/src/openai_proxy/openai.py Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * Change category name from 'GenAI' to 'genai' * Update package requirements with version constraints * Second commit adding notebook * Refactor OpenAI proxy to use base64 encoded script Refactor OpenAI proxy implementation to use base64 encoded script and update FastAPI app configuration. * Change deployment method to OpenAIModule * Third commit adding notebook * Third commit adding notebook * Remove package requirements from item.yaml Removed specific requirements for fastapi and requests. * Rename item and update kind in YAML * Update openai.py * Third commit adding notebook * Fix after review * Fix after review --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * [Evidently] Fill example notebook (#919) * add notebook + rename directory + correct evidently version * remove extra cell * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * [CLI + Modules] Fix time format in generate item yaml script (#922) * fix time format for evidently and hist * fix cli script * fix datetime format * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * Fix CMD first commit * Fix CMD second commit * remove max-width restriction from the main content (#929) * add test, requirement file and notebook * fix cli/utils/helpers.py * [Modules] Modify Evidently & Histogram monitoring apps example notebooks to the change in evaluate() (#934) * histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently dependency * add dependency * remove [ui] from evidently dependency * change notebook name to: openai_proxy_app * [Docs] Add guidelines for contributing new functions or modules (#931) * CONTRIBUTING.md * CONTRIBUTING.md * improvements * [Module] Updated the requirements for OpenAI module (#939) * Updated the requirements * Added Deafult model for credentials * Pin dependencies to specific versions in item.yaml * Update dependencies in openai_proxy_app.py * Specify versions for dependencies in requirements.txt * Fix environment variable access for OPENAI_DEFAULT_MODEL * change notebook name to: openai_proxy_app * [Module] agent_deployer module (#937) * first commit * added all modules * fixes from review and notebook adjustments * added unit tests and documentation * remove log * small fix * cr fixes * get_function fix * cr fixes working deployment * small fix notebook * chore(readme): auto-update asset tables [skip ci] --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: guylei-code <guyleibu@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com> Co-authored-by: Roy Schossberger <85231212+royischoss@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * Chane the vllm-module.ipynb output error ,vllm_app. * Chane the vllm-module.ipynb output error ,vllm_app. * Chane the vllm-module.ipynb output error ,vllm_app. * Remove tensor flag, and fix notebool --------- Co-authored-by: Eyal Danieli <eyald@iguazio.com> Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com> Co-authored-by: Roy Schossberger <85231212+royischoss@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: guylei-code <guyleibu@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com> Co-authored-by: Roy Schossberger <85231212+royischoss@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR: https://iguazio.atlassian.net/browse/FHUB-133.