Skip to content

How to create custom op with fp16 input #23373

Open
@CapJunkrat

Description

Describe the issue

I tried to use custom op functionality to create a custom op (what it does is exactly the same with QuantizeLinear) with fp16 type input, but when I try to inference it, onnxruntime throws error:

onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running HzQuantize node. Name:'HzQuantize_0' Status Message: unknow input type

I checked the function that throws the error, it is "Variadic" initialization function in file "onnxruntime_lite_custom_op.h" , and Float16_t and BFloat16 types are not in the "switch (type)" command, and "ORT_CXX_API_THROW("unknow input type", ORT_RUNTIME_EXCEPTION);" is given from the default switch branch.

Is this done on purpose or just fp16 function not supported yet? Is there a way to give custom op fp16 input type?

To reproduce

N/A

Urgency

No response

Platform

Linux

OS Version

Centos 7

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

latest(cff0ec5)

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions