QUANTIZE silently ignored for unsupported models; precision support not explicit

[Supriya-Krishnamurthi]

Description

When QUANTIZE is provided for models that do not support INT8, the script silently ignores the request and generates only default artifacts (FP32/FP16), while still exiting with status 0.

This leads to a misleading success state, as user intent (INT8 artifacts) is not fulfilled.

Steps to Reproduce

export MODELS_PATH=~/models
/download_public_models.sh centerface coco128

Observed Behavior

FP32/FP16 artifacts are generated
No INT8 artifacts
No warning or error
Exit status = success

Expected Behavior

When QUANTIZE is requested but INT8 is not supported:

Fail with a clear error, or
Emit an explicit warning

Root Cause

• QUANTIZE is only handled in specific model paths (e.g., YOLO)
• Other models (e.g., centerface, yolox_s) ignore it entirely
• missing global validation or post-check for INT8 artifact generation

Suggested Fix

Add a post-check to ensure INT8 artifacts exist when QUANTIZE is set:

if [[ -n "$QUANTIZE" && ! -f "$MODEL_DIR/INT8/$MODEL_NAME.xml" ]]; then
echo "INT8 not supported for $MODEL_NAME"
exit 1
fi

Suggested Improvement

Expose supported precisions per model (FP32/FP16/INT8) instead of implicit behavior, e.g.:

MODEL_PRECISIONS["centerface"]="FP32 FP16"
MODEL_PRECISIONS["yolov8n"]="FP32 FP16 INT8"

This would enable validation, clearer UX, and better maintainability.

[brmarkus]

Have a look into the download script's documentation: https://github.com/open-edge-platform/dlstreamer/blob/d37cef0d2bc773fa87ebcc4bab8b3f438c708855/docs/user-guide/dev_guide/download_public_models.md

It's clear about which models are supported (and which are not, otherwise).

But yeah, a note log message might be helpful.

[Supriya-Krishnamurthi]

Thanks for sharing the documentation.

I see that the supported models for INT8 quantization are clearly listed (YOLOv5u, YOLOv8+, etc.), which helps clarify the intended behavior.

From trying this out, I noticed that when QUANTIZE is provided for unsupported models (e.g., centerface), the script skips quantization and proceeds to generate FP32/FP16 artifacts, eventually exiting successfully without any indication.

This creates a small gap between documentation and runtime behavior:

The documentation defines which models support INT8
But the script doesn’t indicate/validate or communicate when a user requests INT8 for unsupported models
(If there was some error or warning then user would comeback to documentation and check for details but currently it is passing and user expects quantization to be applied)

Just sharing a suggestion here (happy to hear your thoughts):

Emit a warning when QUANTIZE is used with unsupported models
Optionally validate against the documented list and fail early

[Supriya-Krishnamurthi]

@brmarkus I also noticed another related issue, as described below:

When QUANTIZE is used with multiple models (e.g., all or comma-separated lists), some models may support INT8 while others may not. In such cases, the script completes successfully, but it’s not clear which models were actually quantized.

This can make it difficult for users (especially in automation) to verify results.

One possible improvement could be to restrict quantization to a single model, or provide clearer feedback on which models were successfully quantized.

[brmarkus]

Sure, I see the pain.
But would a log message be detected by "the automation"?
Would you treat the whole download (with/without conversion/quantization) as failed when one (the one or one/a few of many) could not be quantized?

Would e.g. a printed summary at the end of the script be helpful, like listing every model, every precision, names, paths? (just thinking about ideas)

[Supriya-Krishnamurthi]

@brmarkus If maintaining a list of supported models and their precisions is not preferred, a runtime-driven approach could make the behavior more predictable and automation-friendly.

Logs alone may not be sufficient, especially since this script is consumed by other projects. Proposing the following design:

1) Use explicit exit codes:

2) Return a structured JSON summary

{ "requested_quantization": true, "int8_success": ["yolo11s", "yolo8n"], "int8_failed": ["centerface"] }

Exit codes provide a quick success/failure signal, while the JSON summary makes the output easy to consume programmatically—especially for multi-model requests—without relying on log parsing.

3) Align downloads with user intent

When INT8 is explicitly requested, downloading FP32/FP16 by default may be unnecessary and increase runtime.

Possible approaches:

Skip downloading other precisions when INT8 is requested, and return an appropriate exit code if INT8 is not generated
Or introduce an option (e.g., --int8-only / --strict-quantize) to enforce INT8-only behavior

Suggested behavior:

• If quantize=coco/coco128 and strict-quantize=true:

  • Only INT8 artifacts are generated, skip downloading other precision artifacts
  • Fail if INT8 is not supported

• Ifquantize=coco/coco128 and strict-quantize=false(default):

  • Allow downloading other supported precisions along with INT8
  • If INT8 is not supported, clearly report it in the JSON summary along with appropriate exit code

I would appreciate your assessment of this approach

[brmarkus]

What about other failure cases (one or some or all downloads failed? ran out of disk space? one or some or all not found (outdated model list) one or some or all dataset failed to retrieve, failed to decompress?)?

[Supriya-Krishnamurthi]

Good point — those are valid general failure cases.

My suggestion is focused specifically on INT8 behavior, not the overall download failures or broader error-handling design of the script. I may not be fully aware of the full flow, so I’ll defer to your judgement on that.

The gap I’m highlighting is that when INT8 is explicitly requested, the script can still succeed without producing INT8 artifacts, and that isn’t clearly reflected to downstream users.