#316 improve task type (#369)

* made task_types only accept transformers tasks
* renamed base_model_spec
* added additional task_type options
* updated docu and changes
* added unit test case with illegal task_type for delete models udf

---------

Co-authored-by: Ariel Schulz <43442541+ArBridgeman@users.noreply.github.com>

Author: MarleneKress79789

doc/changes/unreleased.md

@@ -4,6 +4,8 @@ Code name: T.B.D
 
 ## Summary
 
+T.B.D
+
 ### BREAKING CHANGES:
 
 * The `max_length` parameter has been renamed to `max_new_tokens`, and its behavior changed. Both of these changes where done in accordance with changes in the [transformers library](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.TextGenerationPipeline).
@@ -20,6 +22,21 @@ Code name: T.B.D
 | TE_TOKEN_CLASSIFICATION_UDF                | AI_EXTRACT_EXTENDED         |
 | TE_ZERO_SHOT_CLASSIFICATION_UDF            | AI_CLASSIFY_EXTENDED        |
 
+* `task_type` handling has been changed. 
+The Transformers extension now allows only specific transformers task types in 
+the installation and execution of models.
+You may need to re-install you models from HuggingFace using the new `task_types` in order to use them.
+Models installed with legacy task_types can still be listed and deleted using the respective UDFs.
+
+* Allowed task_types are: 
+             "fill-mask" (previously "filling_mask"),
+             "translation",
+             "zero-shot-classification",
+             "text-classification" (previously "sequence_classification"),
+             "question-answering",
+             "text-generation",
+             "token-classification"
+
 ## Features
 
  * #351: Added functionality for installing default models.
@@ -31,6 +48,7 @@ Code name: T.B.D
 
 ## Security
 
+* Updated urllib3 (2.5.0 -> 2.6.3)
 * Updated exasol-integration-test-docker-environment (4.4.1 -> 5.0.0)
 * Updated exasol-script-languages-container-tool (3.4.1 -> 3.5.0)
 * Updated exasol-saas-api (2.3.0 -> 2.6.0)
@@ -83,3 +101,4 @@ This release fixes vulnerabilities by updating dependencies:
 * #372: Added Transformation Protocol and extracted GetPredictionFromBatch into Transformations
 * #374: Extracted Span handling into Transformations
 * #375: Added implementation for a generalized extract_unique_param_based_dataframes function
+* #316: Changed task_types to only allow transformers task_types, allows underscores and dashes

doc/user_guide/manage_models.md

@@ -104,21 +104,67 @@ SELECT TE_MODEL_DOWNLOADER_UDF(
 * `bucketfs_conn`
 
 Specific parameters
-* `token_conn`: The connection name containing the token required for private models. You can use an empty string ('') for public models. For details on how to create a connection object with token information, please check the [Getting Started](./setup#getting-started) section.
+* `token_conn`: The connection name containing the token required for private models. 
+You can use an empty string ('') for public models. For details on how to create a 
+connection object with token information, please see the 
+[Getting Started](./setup#getting-started) section.
 * `task_type`: See below.
 
 #### Selecting the Task Type
 
-Some models can be used for multiple types of tasks, but Hugging Face Transformers stores different metadata depending on the task of the model, which affects how the model is loaded later. Setting an incorrect task type, or leaving the task type empty may affect the models performance severely.
+Some models can be used for multiple types of tasks, but Hugging Face Transformers 
+stores different metadata depending on the task of the model, which affects how the 
+model is loaded later. Setting an incorrect task type, or leaving the task type empty 
+may affect the model's performance severely.
 
 Available task types are:
-* `filling_mask`
-* `question_answering`
-* `sequence_classification`
-* `text_generation`
-* `token_classification`
-* `translation`
-* `zero_shot_classification`
+
+| task_type                     | UDFs using this task_type   |
+|-------------------------------|-----------------------------|
+| `fill-mask`                   | AI_FILL_MASK_EXTENDED       |
+| `question-answering`          | AI_ANSWER_EXTENDED          |
+| `text-classification`         | AI_CUSTOM_CLASSIFY_EXTENDED |
+| `text-classification`         | AI_ENTAILMENT_EXTENDED      |
+| `text-classification`         | AI_SENTIMENT                |
+| `text-classification`         | AI_ENTAILMENT_EXTENDED      |
+| `text-generation`             | AI_COMPLETE_EXTENDED        |
+| `token-classification`        | AI_EXTRACT_EXTENDED         |
+| `token-classification`        | AI_EXTRACT_ENTITIES         |
+| `translation`                 | AI_TRANSLATE_EXTENDED       |
+| `zero-shot-classification`    | AI_CLASSIFY_EXTENDED        |
+| `zero-shot-classification`    | AI_CLASSIFY                 |
+
+
+Note that you may use underscores (`_`) instead of dashes (`-`).
+
+We also support the installation of models using the following transformers tasks:
+
+* `document-question-answering`
+* `mask-generation`
+* `table-question-answering`
+* `feature-extraction`
+
+However, we do not offer built in UDFs for using these models. So if you need models 
+supporting these tasks, you will need to write your own UDF for running it.
+
+More information about transformers tasks can be found in the [pipeline task parameter description](https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.pipeline.task).
+
+##### Legacy Task Types
+
+We previously supported the followin task_types:
+
+ * `ai_fill_mask_extended`
+ * `filling_mask`
+ * `sequence_classification`
+ * `ai_complete_extended`
+ * `ai_extract_extended`
+
+Models installed with these task types are no longer compatible with our
+prediction UDFs. If you still need these models, you will need to re-install 
+them using the new task-types described above.
+
+However, listing and deleting these models will still work using both the UDF and the Python API.
+
 
 ### Model Uploader Script
 
@@ -164,10 +210,14 @@ Similar to [Store Models in BucketFS](#store-models-in-bucketfs), you have two o
 In order to do this, you might need to find out which models are safed in the Exasol BucketFS. To do this, 
 we provide the `TE_LIST_MODELS_UDF`. See details at the end of this section.
 
+Note: If you installed models using a custom task_type, or a task_type we do 
+not support anymore, you are still able to list and then delete these models using both the UDF and the Python API.
+
 
 ### Delete Model UDF
 
-Using the `TE_DELETE_MODEL_UDF` below, you can delete a model from BucketFS. The parameter values are similar to that one used in [Store Models in BucketFS](#store-models-in-bucketfs).
+Using the `TE_DELETE_MODEL_UDF` below, you can delete a model from BucketFS. 
+The parameter values are similar to that one used in [Store Models in BucketFS](#store-models-in-bucketfs).
 
 Run the UDF with:
 
@@ -223,6 +273,10 @@ for potential error messages, in addition to the input.
 This UDF will fail to return a model if it was saved with the sub_dir parameter empty, 
 or if no config.json file can be found in the model files.
 
+Note: If you installed models using a custom task_type, or a task_type we do 
+not support anymore, you are still able to list these models using the UDF.
+
+
 Call the UDF like this:
 
 ```sql
@@ -233,8 +287,8 @@ SELECT TE_LIST_MODELS_UDF(
 ```
 Example Output:
 
-| BUCKETFS_CONN | SUB_DIR | MODEL_NAME | TASK_NAME | MODEL_PATH               | ERROR_MESSAGE |
+| BUCKETFS_CONN | SUB_DIR | MODEL_NAME | TASK_TYPE | MODEL_PATH               | ERROR_MESSAGE |
 |---------------|---------|------------|-----------|--------------------------|---------------|
-| conn_name     | dir/    | model_name | task_name | dir/model_name_task_name |  None         |
+| conn_name     | dir/    | model_name | task_type | dir/model_name_task_type |  None         |
 | ...           | ...     | ...        | ...       | ...                      |  ...          |
 

exasol_transformers_extension/deployment/default_udf_parameters.py

@@ -40,10 +40,4 @@
         bucketfs_conn_name=DEFAULT_BUCKETFS_CONN_NAME,
         sub_dir=Path(DEFAULT_SUBDIR),
     ),
-    "model_for_another_udf": model_spec_factory.create(
-        model_name="prajjwal1/bert-tiny",
-        task_type="different_task",
-        bucketfs_conn_name=DEFAULT_BUCKETFS_CONN_NAME,
-        sub_dir=Path(DEFAULT_SUBDIR),
-    ),
 }

exasol_transformers_extension/resources/templates/ls_models_udf.jinja.sql

@@ -5,7 +5,7 @@ CREATE OR REPLACE {{ language_alias }} SCALAR SCRIPT "TE_LIST_MODELS_UDF"(
     bucketfs_conn VARCHAR(2000000),
     sub_dir VARCHAR(2000000),
     model_name VARCHAR(2000000),
-    task_name VARCHAR(2000000),
+    task_type VARCHAR(2000000),
     model_path VARCHAR(2000000),
     error_message VARCHAR(2000000) ) AS
 

exasol_transformers_extension/udfs/models/delete_model_udf.py

@@ -46,9 +46,26 @@ def _delete_model(self, ctx) -> tuple[str, str, str, str, bool, str]:
             ctx.task_type,
         )
 
-        current_model_specification = self._current_model_specification_factory.create(
-            model_name, task_type, bucketfs_conn, Path(sub_dir)
-        )  # specifies details of Huggingface model
+        try:
+            current_model_specification = (
+                self._current_model_specification_factory.create(
+                    model_name, task_type, bucketfs_conn, Path(sub_dir)
+                )
+            )  # specifies details of Huggingface model
+        except ValueError as e:
+            # if task_type is not allowed for model_specification, use a placeholder for creation
+            # and then replace using the legacy_set_task_type_from_udf_name.
+            # needed to allow for deletion of already installed models with illegal task_types
+            current_model_specification = (
+                self._current_model_specification_factory.create(
+                    model_name, "fill-mask", bucketfs_conn, Path(sub_dir)
+                )
+            )  # specifies details of Huggingface model
+            current_model_specification.task_type = (
+                current_model_specification.legacy_set_task_type_from_udf_name(
+                    task_type
+                )
+            )
         try:
             # create bucketfs location
             bfs_conn_obj = self._exa.get_connection(bucketfs_conn)

exasol_transformers_extension/udfs/models/ls_models_udf.py

@@ -23,7 +23,7 @@ class ListModelsUDF:
     | directory where models are | BucketFS connection |
 
     returns a table of:
-    bucketfs_conn, sub_dir, model_name, task_name, path of model in BucketFS
+    bucketfs_conn, sub_dir, model_name, task_type, path of model in BucketFS
     """
 
     def __init__(

exasol_transformers_extension/utils/bucketfs_model_specification.py

@@ -20,9 +20,9 @@ def __init__(
             Name of the model. This is the same name as it's seen on the Huggingface
             model card, for example 'cross-encoder/nli-deberta-base'.
         task_type:
-            Name of an NLP task, filling_mask, question_answering,
-            text_classification, text_generation, token_classification,
-            translation, zero_shot_classification.
+            Name of an NLP task, fill-mask, question-answering,
+            text-classification, text-generation, token-classification,
+            translation, zero-shot-classification.
         bucketfs_conn_name:
             Name of the BucketFS connection to retrieve the BucketFS location from.
         sub_dir: