feat: Generalize operator extraction script for unit testing - #257
Open
Zane-Jiang wants to merge 1084 commits into
Open
feat: Generalize operator extraction script for unit testing #257Zane-Jiang wants to merge 1084 commits into
Zane-Jiang wants to merge 1084 commits into
Conversation
- and basic data driven inference Change-Id: Ibb04a3e60a99ebb7313a61fc30390374f7557e3d
- fix no Rope performed for [a,b,c,d]->[-c, -d, a, b] Change-Id: I466ba0271b31f3fafc207115d795bfe84c2f4b54
- Change-Id: Iedb8116689053f028fcb7d8acb2b75e21b4e0b1d
- shape_secs_search_level should be set 1 when strategy 2 - cost_table doesn't fit will cause cluster cost error Change-Id: I1c7adc769a5cb47ee93f3b12d7215b1821942c06
- only count the users in group Change-Id: I465ef0d84cf904b17deefc354e4b7c2d1afb3d41
- new converter for janus Change-Id: If3915953f8b1f62616a596cc1e52f5d6e0b872bf
- correct the variable name and Match_Pattern doc errors Change-Id: I99bff22adac8e6ab9c0e83b53583cdc0428223a9
- Convert negative offset to positive number Change-Id: I055e72c22f72f41773541b9af368a59652d05f08
- fix AttentionOp lowering when HEAD==1 - refactor ReduceFusePattern to handle keepdims==False & -1 axes - fix ReduceFusePattern to skip cross-channel axes because backend not support Change-Id: I4a102f5d54f279897d4347a9b43fe369cd3ea9ca
- disable lgcache when no_save Change-Id: I8a5293603a38b77daf01525c36d8f8ec4a1ad209
- adjust the order of splitting fused_op in dump_calitable Change-Id: Ic1022057ee9996ef5a1c28b61169e8f6766cdcec
- fix ppl PPL_RUNTIME_PATH env in python && release Change-Id: I0a57f83fd439dfc060c3565bc0e6efb1d7741e2e
--Added dimensional degradation steps --Added 1D 2D 3D rearrangement operations --Reset the stride after rearrangement Change-Id: I981856dba956fbb893d9e80216a875990ed3781c
- some models in no_save mode cant be loaded Change-Id: I2e6f244358673fa5e05a818cf055d0c378890351
- suport static and dyn - layergroup issue remained. Dyn mode should --disable_layer_group Change-Id: Ib807deb59515050d94e93b90b73e8fb321044dcf
- make sure dynamic reshape is correct - refine qwen3vl, switch input [4, num_patchs] to [num_patchs, 4] - vit and add use static mode as default Change-Id: Iace854f76507a98eb49be7d9e0788f73fa22d350
- as it will lose high precision in bf16, it should be f32 instead - test by: llm_convert.py -m /workspace/Qwen3-VL-4B-Instruct -s 2048 --max_input_length 1024 --quantize w4bf16 -c bm1688 --out_dir qwen3vl_4b_debug --max_pixels 768,768 Change-Id: If85712cefa86a6afc2e4350a282534c77e3aa6b2
- cv184x doesnt support f32 Change-Id: I2e7a477f675c419bc99a62cf6d1d4bb18de1a42e
- bm1684 not support layergroup multi-branch case for StoreOp Change-Id: Ia18c6322aed5310ae21d36d602d002e46991871f
…move redundant sensitive layer search - remove some print - better to run mix precision version of int8 model for more reliable accuracy - sensitive layer search is redundant to search qtable - avoid mem leak in search qtable Change-Id: If22930cc7c766770714d29e63bcb22939d287c0d
- Change-Id: Iec7c1ac51df8cd9355d2c7949fc7bf7e4c203e56
- remove duplication and add s_p layers in qtable Change-Id: I31db9b87fae46d34b04d85e2fe72e0540b18da5f
-fix some bcbinary rerrangement logic bug Change-Id: I10aa3098730ef158a9ca0c405764ef69752a9336
- set the right cali_table name when tune_num=0 Change-Id: Ic16fd837ea18d467d04ffade707e445b1b5d3b49
- if keep_dims, input_shape==output_shape; else input= [1, M, q, d], output = [1, M, q*d] - if awq, force to w4f16 Change-Id: I190ec09774fd508a2fadaf499092daa45e9b6d49
- quant.drawio is the diagram file of quant.png Change-Id: Ida1fc099feafc71db8523ee9ba8a2692781ca3f6
…ions - The interp op does not currently support interpolation in the D dimension - 5-dimensional DHW dimension interp is transformed into the last 3-dimensional Gridsampler op Change-Id: If6532cfa747ec8bf904d4d43ba15cf4827b4712d
- update mars3 backend fix the bug of global depthwise Deconv Change-Id: I8784435956e8a319fea8e76af3217f44c348a0a0
- as said Change-Id: Ic7e712a70639c4500a066658a37cf76602ec21a8
- concat addr opt wont be applied in io_alone mode - allocatormethod adds io addr detect Change-Id: Ib797ef407bd053594a086783938c67d80673631d
- IO_ALONE use one tag Change-Id: I6a1e6d591eb042586417737ceb8092e03d9514eb
- fix onnx_convert for clip_op param - disable local reshape_op in 3D group. Change-Id: Iafee7fe01b422e9dd0576a1d9d254a76ba8ac3bf
- bugfix for top canno pass of scaleop Change-Id: I2fe99f969e865087102a76e5bde93cba70c020f5
- split llm bmodels by multi device with pipeline Change-Id: I705af8ea50bc4418f5b4811b324204b9ff0ba4c8
-Align BM1684X MatMul requant round_mode with MLIR for fuse_rq=false Change-Id: I55820e71b9bc73b0bc52a53a3d97bb606f96ce84
- bad logic to invoke this function Change-Id: I07de1d42b48a8b4f8c9c081971af1f36f411f764
- add permuteOp, sliceOp, reshapeOp affinemap for load - add permuteOp, sliceOp, reshapeOp, padOp affinemap for store - optimize codegen cost to seconds Change-Id: I467377710ac44714c851446000a531bae615b33b
- Add qwen_asr model_type to conver the torch model Change-Id: I9a33e30a057e5c28259e41a16c819ded0b007850
- add kernel module - add ppl complie - fix interp op Change-Id: I1a9914a69aab5ce5d4cba122a1fc5ffe651c8ee8
- fix type judgement error Change-Id: Ia6e2ca94badb71d0a7f2727e2a845cba4a898455
- change indices from fp32 to int32 - change op type from fp32 to int 8 Change-Id: I2f28f6ab6fe3dc8d8cf356392b75ea4788728333
- support roma top/tpu.mlir cuda inference Change-Id: I6dbed58c78b37ca5f3a57134b58c3fcece575a40
- infer kernel size from weight when onnx conv node miss kernel shape attr Change-Id: Ifdabcd7fb2675fd9a68a29d6f114e265e73737fb
- CanCutGridSamplerFusePattern supports int8 quantization, Fix the correlation matching logic Change-Id: I2fdf438fd3ab9055eafe2cfd6590c6cbe88a85c1
- test bm1684x/qwen1.5-moe Change-Id: I854ef77718e1951d83c74b38968e8b41edb3a9aa
- allow the second slice to start from start and the penultimate slice to reach the end - enable dwconv backward propagation on the BM1684X family Change-Id: Id86e4bed61841de82c41d376838090f40a68e713
- test by: llm_convert.py -m /workspace/Qwen3.5/Qwen3.5-4B-int4-AutoRound -s 2048 --max_input_length 1024 -c bm1684x --out_dir qwen3.5 --max_pixels 768,768 --debug - add chunk_gated_delta_rule by da.teng - fix some convert errors Change-Id: Ic6cf2c239b6c32cbdca3159b87fd8c1ac78f507a
- [SOPHONSILK-581] Change-Id: I53cccb52f8a52c40e4fe6f57f3ac6183f3624596
- add more op support Change-Id: Id0067663f45bc0e438ea7c589b6fa2f2d3604623
- when outputs has the same addr with inputs, addr assign will assign more addr Change-Id: Ifb0af3959ca8c372b24dd009026d242ceea66add
- make it more clear Change-Id: I9ca7c9c5af98effcc59a44aaf83901dea29b4c33
- set eu_align=false in LayerGroup for RMSNorm Change-Id: If242ef9bd2229f1f6114d3d5236632b5df6b2ae3
- fix readme, add requirements, etc - update info and fix compiling WARNINGs in docs Change-Id: Ia24a5c789bc11c29bc09bdb090b31993c521a5d7
- renew README.md Change-Id: Id6591f9a3a91c3ec98ba51c12cbcf88075cc2b81
- missing group info Change-Id: I4bfdbcf689826d96c445cefb3ddd195255ce16c7
- remove unnecessary code in store - support offset < 0 for slice in affine opt Change-Id: I006cc29559f00241d4e253084dc55df101541f35
- use embed_vit/block_{i}/lmead names for pp bmodels
Change-Id: Ib178ca5629abf3c3870c319798ab2c09e78b9295
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
18 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this pr do?
Fix issue #238
Implement onnx/torch operator extraction in extract_op.py, used to extract operator case shapes and parameters from MLIR files into a JSON file.
Add options
--json_fileand--case_idto test_onnx.py to load JSON files as test case inputs for operator testing.Only a subset of basic and commonly used operators is supported.
Usage:
1.extract
extract_op.py --framework {onnx,torch} --op {Abs,Add,Sub,Mul,Div,AddConst,And,Relu,Concat,Compare,Softmax} mlir_file output_json #eg extract_op.py onnx_test_bm1684x/Add/Add.mlir op_add.json --framework=onnx --op=Add2.test
test_onnx.py --json_file=xxx --case_id=0 --case=xxx #eg test_onnx.py --json_file=op_add.json --case_id=0 --case=Add