Commit 5ed7d41
authored
[GPU] Enable INT4 KV-cache compress (#33109)
### Description of the issue
- Enable 4bit KV-cache compression for peak memory reduction
- Pass WWB accuracy check, no performance regression by this feature
#### Reproduction step and snapshot
- Run WWB test with ov-config to enable 4bit KV-cache
`--ov-config enable_u4.json`
- sample json file :
`{
"KV_CACHE_PRECISION": "u4",
"KEY_CACHE_QUANT_MODE": "BY_CHANNEL"
}`
- execution WWB
`python wwb.py --genai --ov-config enable_uu4.json --target-model
minicpm4-0.5b/pytorch/ov/OV_FP16-INT8_ASYM --device GPU --gt-data
gt-data/minicpm4-0.5b__NAT/reference.csv"`
#### Checklist
- [x] Is it a proper fix?
- [x] Did you include test case for this fix, if necessary?
- [x] Did you review existing test that can be extended to cover this
scenario?
### AI Assistance:
- AI assistance used: yes
Summarized validation report.
Analyzed failed unt-test cases and generated proper solution.
### Tickets:
- CVS-169489, CVS-180645
---------
Signed-off-by: Min, Byungil <byungil.min@intel.com>1 parent 5884aeb commit 5ed7d41
File tree
11 files changed
+1024
-170
lines changed- src
- common/transformations
- include/transformations/common_optimizations
- src/transformations/common_optimizations
- tests/common_optimizations
- plugins/intel_gpu
- src
- graph
- impls/ocl_v2
- sdpa
- kernel_selector/cl_kernels/include/batch_headers
- plugin
- transformations
- tests/unit/test_cases
11 files changed
+1024
-170
lines changedLines changed: 5 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
39 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
| |||
45 | 48 | | |
46 | 49 | | |
47 | 50 | | |
| 51 | + | |
48 | 52 | | |
49 | 53 | | |
50 | 54 | | |
| |||
Lines changed: 14 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
26 | 28 | | |
27 | | - | |
| 29 | + | |
| 30 | + | |
28 | 31 | | |
29 | 32 | | |
30 | 33 | | |
| |||
87 | 90 | | |
88 | 91 | | |
89 | 92 | | |
| 93 | + | |
90 | 94 | | |
91 | 95 | | |
92 | 96 | | |
| |||
105 | 109 | | |
106 | 110 | | |
107 | 111 | | |
| 112 | + | |
108 | 113 | | |
109 | 114 | | |
110 | 115 | | |
| |||
147 | 152 | | |
148 | 153 | | |
149 | 154 | | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
150 | 162 | | |
151 | 163 | | |
152 | 164 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
351 | 351 | | |
352 | 352 | | |
353 | 353 | | |
| 354 | + | |
354 | 355 | | |
355 | 356 | | |
356 | 357 | | |
| |||
0 commit comments