Skip to content

Commit 8d66ed1

Browse files
committed
update flowchart
Signed-off-by: Prathamesh Mandke <46148373+pkmandke@users.noreply.github.com>
1 parent 0f51025 commit 8d66ed1

2 files changed

Lines changed: 11 additions & 11 deletions

File tree

docs/src/quantization/overview.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -204,7 +204,7 @@ The two modes are expected to produce very similar models for weight-only quanti
204204
A few scenarios where `eager` mode may need to be used instead of `graph`:
205205

206206
- If you run into any errors during the `prepare` call which, under the hood, invokes the `torch.export.export` and `torchao`'s `prepare_qat_pt2e`/`convert_pt2e` APIs. See [Graph Mode Troubleshooting](../debugging/graph_mode_troubleshooting.md) for common export errors and workarounds before falling back to eager mode.
207-
- When `torch.nn.Module` needs to be provided as an input, instead of `ExportedProgram` to the conversion API of [coreai-torch](https://github.com/apple/coreai-torch). Note that models whose submodules must be "externalized" to map them to _composite ops_ for better runtime performance can still be quantized in graph mode. See [Quantizing Models with Core AI Composite Ops](../utils/composite_op_quantization.md).
207+
- When `torch.nn.Module` needs to be provided as an input, instead of `ExportedProgram` to the conversion API of [coreai-torch](https://github.com/apple/coreai-torch). Note that models whose submodules must be "externalized" to map them to _composite ops_ for better runtime performance can still be quantized in graph mode. See [Quantizing Models with Core AI Composite Ops in Graph Mode](../utils/composite_op_quantization.md).
208208

209209
#### Weights and activations quantization
210210

docs/src/utils/composite_op_quantization.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Quantizing Models with Core AI Composite Ops
1+
# Quantizing Models with Core AI Composite Ops in Graph Mode
22

33
Core AI recognizes certain well-known building blocks, such as SDPA or RMSNorm, as _composite ops_ and applies optimized implementations for them.
44
`coreai-torch` establishes those boundaries through _externalization_.
@@ -20,17 +20,17 @@ The externalization APIs used below, `_patch_model_for_externalization` and `_su
2020

2121
```mermaid
2222
---
23-
title: "Composite Op Quantization Workflow"
23+
title: "Graph mode Quantization Workflow with Externalization"
2424
---
2525
flowchart LR
26-
model["Full Precision<br>Model"] --> patch["Patch model"]
27-
patch --> prepare["quantizer.prepare(...)"]
28-
prepare --> calibrate["Calibrate"]
29-
calibrate --> qfin["quantizer.finalize(...)"]
30-
qfin --> export["torch.export.export(...)"]
31-
export --> sub["Sub-export<br>and restore"]
32-
sub --> convert["TorchConverter().to_coreai()"]
26+
model["Full Precision<br>Model"] --> patch["Patch Model for<br>Externalization"]
27+
patch --> prepare["Prepare and<br>Calibrate"]
28+
prepare --> qfin["Finalize and<br>Export"]
29+
qfin --> sub["Sub-export<br>and Restore"]
30+
sub --> convert["Convert to<br>Core AI"]
3331
style model fill:#f9f9f9,stroke:#999
32+
style patch fill:#e8f0fe,stroke:#4285f4
33+
style sub fill:#e8f0fe,stroke:#4285f4
3434
```
3535

3636
## Step 1: Patch the model before prepare
@@ -239,4 +239,4 @@ coreai.graph @main(%arg0: tensor<1x32xf32> {coreai.name = "x"}) -> (tensor<1x32x
239239

240240
## Notes
241241

242-
- The same set of APIs and workflows can be followed for Quantization Aware Training in `graph` mode as well.
242+
- The same set of APIs and steps apply for Quantization Aware Training in `graph` mode as well.

0 commit comments

Comments
 (0)