Skip to content

有相关onnx模型转换与调用的支持嘛? #172

@Gmgge

Description

@Gmgge

你好,非常感谢你们开源如此棒的工作

我在尝试将模型转换为onnx,以使用推理框架onnxruntime。
我进行了以下的工作
1.参考huggingface/transformers,调用模型转换将预训练模型转换为onnx,参考教程在此
运行脚本为:
python -m transformers.onnx --model "/project/gm/project/transformers/albert-chinese-tiny-train/" albert-chinese-tiny-train
结果为:
Some weights of the model checkpoint at /project/gm/project/transformers/albert-chinese-tiny-train/ were not used when initializing AlbertModel: ['predictions.dense.bias', 'predics.decoder.bias', 'predictions.LayerNorm.weight', 'predictions.LayerNorm.bias', 'predictions.decoder.weight', 'predictions.dense.weight']

  • This IS expected if you are initializing AlbertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClasBertForPreTraining model).
  • This IS NOT expected if you are initializing AlbertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification modeClassification model).
    Some weights of AlbertModel were not initialized from the model checkpoint at /project/gm/project/transformers/albert-chinese-tiny-train/ and are newly initialized: ['albert.pooler.bias']
    You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
    Using framework PyTorch: 1.8.1+cu102
    Validating ONNX model...
    -[✓] ONNX model output names match reference model ({'last_hidden_state'})
    - Validating ONNX Model output "last_hidden_state":
    -[✓] (2, 8, 312) matches (2, 8, 312)
    -[✓] all values close (atol: 1e-05)
    All good, model saved at: albert-chinese-tiny-train/model.onnx

2.参考transformers中对onnx的模型调用,结果提示参数错误
session = InferenceSession("onnx/model.onnx")
inputs = tokenizer("Using DistilBERT with ONNX Runtime!", return_tensors="np")
outputs = session.run(output_names=["last_hidden_state"], input_feed=dict(inputs))

有相关提示或者文档嘛?我阅读了本项目的文档,但是没有找到相关信息,不知道是否因为有些遗漏。

如果有相关的提示或者样例代码以参考,非常感谢

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions