Skip to content

Commit f086964

Browse files
Eyal-Danielidanielperezziguazio-cicdguylei-codeamitnGiniApps
authored
Cherry Pick from Development (#936)
* replace author to Iguazio manually (#905) * Organize CLI directory + new CLI for generating item.yaml files (#906) * create a CLI for generating item.yaml and organize the CLI directory * modify comments to module * PR fixes * Update cli/common/generate_item_yaml.py Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> --------- Co-authored-by: Eyal Danieli <eyal_danieli@mckinsey.com> * fill count events notebook (#908) * avoid noise reduction unit test (#909) * Add histogram-data-drift monitoring application module (without example) (#911) * histogram data drift module with empty example notebook * post review fixes * chore(readme): auto-update asset tables [skip ci] * Fill histogram-data-drift example notebook (#912) * fill data-drift nb * post review fixes * Add evidently demo app monitoring application module (without example) (#913) * sphinx build docs bug fix * add evidently demo app module (empty example notebook) * post review changes * chore(readme): auto-update asset tables [skip ci] * [Translate] Require torch>=2.6 for the translate function to work properly (#915) * lock torch valid version * edit the item.yaml and generated function.yaml * update mlrun version * [CLI] Generated READMEs are produced with broken links to the items (#918) * fix * test fix * test fix * test fix * test fix * final workflow * chore(readme): auto-update asset tables [skip ci] * OpenAI Module without notebook (#917) * First commit OpenAI Module * First commit OpenAI Module * Update example filename in item.yaml * Delete modules/src/openai_proxy/requirements.txt No need due to no unitest * Update item.yaml for OpenAI application configuration * Update modules/src/openai_proxy/openai.py Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * Change category name from 'GenAI' to 'genai' * Update package requirements with version constraints * Second commit adding notebook * Refactor OpenAI proxy to use base64 encoded script Refactor OpenAI proxy implementation to use base64 encoded script and update FastAPI app configuration. * Change deployment method to OpenAIModule * Third commit adding notebook * Third commit adding notebook * Remove package requirements from item.yaml Removed specific requirements for fastapi and requests. * Rename item and update kind in YAML * Update openai.py * Third commit adding notebook * Fix after review * Fix after review --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> * chore(readme): auto-update asset tables [skip ci] * [Evidently] Fill example notebook (#919) * add notebook + rename directory + correct evidently version * remove extra cell * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * [CLI + Modules] Fix time format in generate item yaml script (#922) * fix time format for evidently and hist * fix cli script * fix datetime format * chore(readme): auto-update asset tables [skip ci] * chore(readme): auto-update asset tables [skip ci] * Fix CMD first commit * Fix CMD second commit * remove max-width restriction from the main content (#929) * add test, requirement file and notebook * fix cli/utils/helpers.py * [Modules] Modify Evidently & Histogram monitoring apps example notebooks to the change in evaluate() (#934) * histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * fix to histogram_data_drift.ipynb * evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently_iris.ipynb * fix evidently dependency * add dependency * remove [ui] from evidently dependency * change notebook name to: openai_proxy_app --------- Co-authored-by: Daniel Perez <100069700+danielperezz@users.noreply.github.com> Co-authored-by: iguazio-cicd <iguaziocicd@gmail.com> Co-authored-by: guylei-code <guyleibu@gmail.com> Co-authored-by: amitnGiniApps <amitn@gini-apps.com>
1 parent ca5f9a0 commit f086964

File tree

13 files changed

+396
-440
lines changed

13 files changed

+396
-440
lines changed

‎cli/marketplace/_static/css/custom.css‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,3 +47,9 @@ html:root {
4747
position: relative;
4848
box-shadow: none !important;
4949
}
50+
51+
/* Remove max-width from the content */
52+
.bd-main .bd-content .bd-article-container {
53+
max-width: none;
54+
width: 100%;
55+
}

‎cli/utils/helpers.py‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -90,12 +90,12 @@ def install_python(directory: Union[str, Path]):
9090
return python_location
9191

9292

93-
def _run_subprocess(cmd: str, directory):
93+
def _run_subprocess(cmd: List[str], directory):
9494
completed_process: subprocess.CompletedProcess = subprocess.run(
9595
cmd,
9696
stdout=sys.stdout,
9797
stderr=subprocess.PIPE,
98-
shell=True,
98+
shell=False,
9999
cwd=directory,
100100
)
101101
exit_on_non_zero_return(completed_process)
@@ -118,9 +118,8 @@ def install_requirements(
118118

119119
if requirements_file.exists():
120120
print(f"Installing requirements from {requirements_file}...")
121-
_run_subprocess(
122-
f"pipenv install --skip-lock -r {requirements_file}", directory
123-
)
121+
cmd = ["pipenv", "install", "--skip-lock", "-r", str(requirements_file)]
122+
_run_subprocess(cmd, directory)
124123
with open(requirements_file, "r") as f:
125124
mlrun_version = [l.replace("\n", "") for l in f.readlines() if "mlrun" in l]
126125
# remove mlrun from requirements if installed with version limits:
@@ -129,9 +128,8 @@ def install_requirements(
129128

130129
if requirements:
131130
print(f"Installing requirements [{' '.join(requirements)}] for {directory}...")
132-
_run_subprocess(
133-
f"pipenv install --skip-lock {' '.join(requirements)}", directory
134-
)
131+
cmd = ["pipenv", "install", "--skip-lock", *requirements]
132+
_run_subprocess(cmd, directory)
135133

136134

137135
def get_item_yaml_values(

‎functions/README.md‎

Lines changed: 36 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -9,40 +9,40 @@ it is expected that contributors follow certain guidelines/protocols (please chi
99
<!-- AUTOGEN:START (do not edit below) -->
1010
| Name | Description | Kind | Categories |
1111
| --- | --- | --- | --- |
12-
| [aggregate](https://github.com/mlrun/functions/tree/master/functions/src/aggregate) | Rolling aggregation over Metrics and Lables according to specifications | job | data-preparation |
13-
| [arc_to_parquet](https://github.com/mlrun/functions/tree/master/functions/src/arc_to_parquet) | retrieve remote archive, open and save as parquet | job | utils |
14-
| [auto_trainer](https://github.com/mlrun/functions/tree/master/functions/src/auto_trainer) | Automatic train, evaluate and predict functions for the ML frameworks - Scikit-Learn, XGBoost and LightGBM. | job | machine-learning, model-training |
15-
| [azureml_serving](https://github.com/mlrun/functions/tree/master/functions/src/azureml_serving) | AzureML serving function | serving | machine-learning, model-serving |
16-
| [azureml_utils](https://github.com/mlrun/functions/tree/master/functions/src/azureml_utils) | Azure AutoML integration in MLRun, including utils functions for training models on Azure AutoML platfrom. | job | model-serving, utils |
17-
| [batch_inference](https://github.com/mlrun/functions/tree/master/functions/src/batch_inference) | Batch inference (also knows as prediction) for the common ML frameworks (SciKit-Learn, XGBoost and LightGBM) while performing data drift analysis. | job | model-serving |
18-
| [batch_inference_v2](https://github.com/mlrun/functions/tree/master/functions/src/batch_inference_v2) | Batch inference (also knows as prediction) for the common ML frameworks (SciKit-Learn, XGBoost and LightGBM) while performing data drift analysis. | job | model-serving |
19-
| [describe](https://github.com/mlrun/functions/tree/master/functions/src/describe) | describe and visualizes dataset stats | job | data-analysis |
20-
| [describe_dask](https://github.com/mlrun/functions/tree/master/functions/src/describe_dask) | describe and visualizes dataset stats | job | data-analysis |
21-
| [describe_spark](https://github.com/mlrun/functions/tree/master/functions/src/describe_spark) | | job | data-analysis |
22-
| [feature_selection](https://github.com/mlrun/functions/tree/master/functions/src/feature_selection) | Select features through multiple Statistical and Model filters | job | data-preparation, machine-learning |
23-
| [gen_class_data](https://github.com/mlrun/functions/tree/master/functions/src/gen_class_data) | Create a binary classification sample dataset and save. | job | data-generation |
24-
| [github_utils](https://github.com/mlrun/functions/tree/master/functions/src/github_utils) | add comments to github pull request | job | utils |
25-
| [hugging_face_serving](https://github.com/mlrun/functions/tree/master/functions/src/hugging_face_serving) | Generic Hugging Face model server. | serving | genai, model-serving |
26-
| [load_dataset](https://github.com/mlrun/functions/tree/master/functions/src/load_dataset) | load a toy dataset from scikit-learn | job | data-preparation |
27-
| [mlflow_utils](https://github.com/mlrun/functions/tree/master/functions/src/mlflow_utils) | Mlflow model server, and additional utils. | serving | model-serving, utils |
28-
| [model_server](https://github.com/mlrun/functions/tree/master/functions/src/model_server) | generic sklearn model server | nuclio:serving | model-serving, machine-learning |
29-
| [model_server_tester](https://github.com/mlrun/functions/tree/master/functions/src/model_server_tester) | test model servers | job | monitoring, model-serving |
30-
| [noise_reduction](https://github.com/mlrun/functions/tree/master/functions/src/noise_reduction) | Reduce noise from audio files | job | data-preparation, audio |
31-
| [onnx_utils](https://github.com/mlrun/functions/tree/master/functions/src/onnx_utils) | ONNX intigration in MLRun, some utils functions for the ONNX framework, optimizing and converting models from different framework to ONNX using MLRun. | job | utils, deep-learning |
32-
| [open_archive](https://github.com/mlrun/functions/tree/master/functions/src/open_archive) | Open a file/object archive into a target directory | job | utils |
33-
| [pii_recognizer](https://github.com/mlrun/functions/tree/master/functions/src/pii_recognizer) | This function is used to recognize PII in a directory of text files | job | data-preparation, NLP |
34-
| [pyannote_audio](https://github.com/mlrun/functions/tree/master/functions/src/pyannote_audio) | pyannote's speech diarization of audio files | job | deep-learning, audio |
35-
| [question_answering](https://github.com/mlrun/functions/tree/master/functions/src/question_answering) | GenAI approach of question answering on a given data | job | genai |
36-
| [send_email](https://github.com/mlrun/functions/tree/master/functions/src/send_email) | Send Email messages through SMTP server | job | utils |
37-
| [silero_vad](https://github.com/mlrun/functions/tree/master/functions/src/silero_vad) | Silero VAD (Voice Activity Detection) functions. | job | deep-learning, audio |
38-
| [sklearn_classifier](https://github.com/mlrun/functions/tree/master/functions/src/sklearn_classifier) | train any classifier using scikit-learn's API | job | machine-learning, model-training |
39-
| [sklearn_classifier_dask](https://github.com/mlrun/functions/tree/master/functions/src/sklearn_classifier_dask) | train any classifier using scikit-learn's API over Dask | job | machine-learning, model-training |
40-
| [structured_data_generator](https://github.com/mlrun/functions/tree/master/functions/src/structured_data_generator) | GenAI approach of generating structured data according to a given schema | job | data-generation, genai |
41-
| [test_classifier](https://github.com/mlrun/functions/tree/master/functions/src/test_classifier) | test a classifier using held-out or new data | job | machine-learning, model-testing |
42-
| [text_to_audio_generator](https://github.com/mlrun/functions/tree/master/functions/src/text_to_audio_generator) | Generate audio file from text using different speakers | job | data-generation, audio |
43-
| [tf2_serving](https://github.com/mlrun/functions/tree/master/functions/src/tf2_serving) | tf2 image classification server | nuclio:serving | model-serving, machine-learning |
44-
| [transcribe](https://github.com/mlrun/functions/tree/master/functions/src/transcribe) | Transcribe audio files into text files | job | audio, genai |
45-
| [translate](https://github.com/mlrun/functions/tree/master/functions/src/translate) | Translate text files from one language to another | job | genai, NLP |
46-
| [v2_model_server](https://github.com/mlrun/functions/tree/master/functions/src/v2_model_server) | generic sklearn model server | serving | model-serving, machine-learning |
47-
| [v2_model_tester](https://github.com/mlrun/functions/tree/master/functions/src/v2_model_tester) | test v2 model servers | job | model-testing, machine-learning |
12+
| [aggregate](https://github.com/mlrun/functions/tree/development/functions/src/aggregate) | Rolling aggregation over Metrics and Lables according to specifications | job | data-preparation |
13+
| [arc_to_parquet](https://github.com/mlrun/functions/tree/development/functions/src/arc_to_parquet) | retrieve remote archive, open and save as parquet | job | utils |
14+
| [auto_trainer](https://github.com/mlrun/functions/tree/development/functions/src/auto_trainer) | Automatic train, evaluate and predict functions for the ML frameworks - Scikit-Learn, XGBoost and LightGBM. | job | machine-learning, model-training |
15+
| [azureml_serving](https://github.com/mlrun/functions/tree/development/functions/src/azureml_serving) | AzureML serving function | serving | machine-learning, model-serving |
16+
| [azureml_utils](https://github.com/mlrun/functions/tree/development/functions/src/azureml_utils) | Azure AutoML integration in MLRun, including utils functions for training models on Azure AutoML platfrom. | job | model-serving, utils |
17+
| [batch_inference](https://github.com/mlrun/functions/tree/development/functions/src/batch_inference) | Batch inference (also knows as prediction) for the common ML frameworks (SciKit-Learn, XGBoost and LightGBM) while performing data drift analysis. | job | model-serving |
18+
| [batch_inference_v2](https://github.com/mlrun/functions/tree/development/functions/src/batch_inference_v2) | Batch inference (also knows as prediction) for the common ML frameworks (SciKit-Learn, XGBoost and LightGBM) while performing data drift analysis. | job | model-serving |
19+
| [describe](https://github.com/mlrun/functions/tree/development/functions/src/describe) | describe and visualizes dataset stats | job | data-analysis |
20+
| [describe_dask](https://github.com/mlrun/functions/tree/development/functions/src/describe_dask) | describe and visualizes dataset stats | job | data-analysis |
21+
| [describe_spark](https://github.com/mlrun/functions/tree/development/functions/src/describe_spark) | | job | data-analysis |
22+
| [feature_selection](https://github.com/mlrun/functions/tree/development/functions/src/feature_selection) | Select features through multiple Statistical and Model filters | job | data-preparation, machine-learning |
23+
| [gen_class_data](https://github.com/mlrun/functions/tree/development/functions/src/gen_class_data) | Create a binary classification sample dataset and save. | job | data-generation |
24+
| [github_utils](https://github.com/mlrun/functions/tree/development/functions/src/github_utils) | add comments to github pull request | job | utils |
25+
| [hugging_face_serving](https://github.com/mlrun/functions/tree/development/functions/src/hugging_face_serving) | Generic Hugging Face model server. | serving | genai, model-serving |
26+
| [load_dataset](https://github.com/mlrun/functions/tree/development/functions/src/load_dataset) | load a toy dataset from scikit-learn | job | data-preparation |
27+
| [mlflow_utils](https://github.com/mlrun/functions/tree/development/functions/src/mlflow_utils) | Mlflow model server, and additional utils. | serving | model-serving, utils |
28+
| [model_server](https://github.com/mlrun/functions/tree/development/functions/src/model_server) | generic sklearn model server | nuclio:serving | model-serving, machine-learning |
29+
| [model_server_tester](https://github.com/mlrun/functions/tree/development/functions/src/model_server_tester) | test model servers | job | monitoring, model-serving |
30+
| [noise_reduction](https://github.com/mlrun/functions/tree/development/functions/src/noise_reduction) | Reduce noise from audio files | job | data-preparation, audio |
31+
| [onnx_utils](https://github.com/mlrun/functions/tree/development/functions/src/onnx_utils) | ONNX intigration in MLRun, some utils functions for the ONNX framework, optimizing and converting models from different framework to ONNX using MLRun. | job | utils, deep-learning |
32+
| [open_archive](https://github.com/mlrun/functions/tree/development/functions/src/open_archive) | Open a file/object archive into a target directory | job | utils |
33+
| [pii_recognizer](https://github.com/mlrun/functions/tree/development/functions/src/pii_recognizer) | This function is used to recognize PII in a directory of text files | job | data-preparation, NLP |
34+
| [pyannote_audio](https://github.com/mlrun/functions/tree/development/functions/src/pyannote_audio) | pyannote's speech diarization of audio files | job | deep-learning, audio |
35+
| [question_answering](https://github.com/mlrun/functions/tree/development/functions/src/question_answering) | GenAI approach of question answering on a given data | job | genai |
36+
| [send_email](https://github.com/mlrun/functions/tree/development/functions/src/send_email) | Send Email messages through SMTP server | job | utils |
37+
| [silero_vad](https://github.com/mlrun/functions/tree/development/functions/src/silero_vad) | Silero VAD (Voice Activity Detection) functions. | job | deep-learning, audio |
38+
| [sklearn_classifier](https://github.com/mlrun/functions/tree/development/functions/src/sklearn_classifier) | train any classifier using scikit-learn's API | job | machine-learning, model-training |
39+
| [sklearn_classifier_dask](https://github.com/mlrun/functions/tree/development/functions/src/sklearn_classifier_dask) | train any classifier using scikit-learn's API over Dask | job | machine-learning, model-training |
40+
| [structured_data_generator](https://github.com/mlrun/functions/tree/development/functions/src/structured_data_generator) | GenAI approach of generating structured data according to a given schema | job | data-generation, genai |
41+
| [test_classifier](https://github.com/mlrun/functions/tree/development/functions/src/test_classifier) | test a classifier using held-out or new data | job | machine-learning, model-testing |
42+
| [text_to_audio_generator](https://github.com/mlrun/functions/tree/development/functions/src/text_to_audio_generator) | Generate audio file from text using different speakers | job | data-generation, audio |
43+
| [tf2_serving](https://github.com/mlrun/functions/tree/development/functions/src/tf2_serving) | tf2 image classification server | nuclio:serving | model-serving, machine-learning |
44+
| [transcribe](https://github.com/mlrun/functions/tree/development/functions/src/transcribe) | Transcribe audio files into text files | job | audio, genai |
45+
| [translate](https://github.com/mlrun/functions/tree/development/functions/src/translate) | Translate text files from one language to another | job | genai, NLP |
46+
| [v2_model_server](https://github.com/mlrun/functions/tree/development/functions/src/v2_model_server) | generic sklearn model server | serving | model-serving, machine-learning |
47+
| [v2_model_tester](https://github.com/mlrun/functions/tree/development/functions/src/v2_model_tester) | test v2 model servers | job | model-testing, machine-learning |
4848
<!-- AUTOGEN:END -->

‎modules/README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@
66
<!-- AUTOGEN:START (do not edit below) -->
77
| Name | Description | Kind | Categories |
88
| --- | --- | --- | --- |
9-
| [count_events](https://github.com/mlrun/functions/tree/master/modules/src/count_events) | Count events in each time window | monitoring_application | model-serving |
10-
| [evidently_iris](https://github.com/mlrun/functions/tree/master/modules/src/evidently_iris) | Demonstrates Evidently integration in MLRun for data quality and drift monitoring using the Iris dataset | monitoring_application | model-serving, structured-ML |
11-
| [histogram_data_drift](https://github.com/mlrun/functions/tree/master/modules/src/histogram_data_drift) | Model-monitoring application for detecting and visualizing data drift | monitoring_application | model-serving, structured-ML |
12-
| [openai_proxy_app](https://github.com/mlrun/functions/tree/master/modules/src/openai_proxy_app) | OpenAI application runtime based on fastapi | generic | genai |
9+
| [count_events](https://github.com/mlrun/functions/tree/development/modules/src/count_events) | Count events in each time window | monitoring_application | model-serving |
10+
| [evidently_iris](https://github.com/mlrun/functions/tree/development/modules/src/evidently_iris) | Demonstrates Evidently integration in MLRun for data quality and drift monitoring using the Iris dataset | monitoring_application | model-serving, structured-ML |
11+
| [histogram_data_drift](https://github.com/mlrun/functions/tree/development/modules/src/histogram_data_drift) | Model-monitoring application for detecting and visualizing data drift | monitoring_application | model-serving, structured-ML |
12+
| [openai_proxy_app](https://github.com/mlrun/functions/tree/development/modules/src/openai_proxy_app) | OpenAI application runtime based on fastapi | generic | genai |
1313
<!-- AUTOGEN:END -->

0 commit comments

Comments
 (0)