Skip to content

Commit 127e46e

Browse files
ashotikashotik
committed
doc update
1 parent 75eca38 commit 127e46e

28 files changed

Lines changed: 33 additions & 58 deletions

‎docs/doc.html‎

Lines changed: 33 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -138,23 +138,8 @@
138138
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Though a
139139
2-billion-parameter model is small, it still requires 4 to 8 GB of memory.</span></p>
140140

141-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
142-
143-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>1. Using
144-
float32 (4 bytes per parameter):</span></p>
145-
146-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>2,000,000,000
147-
parameters × 4 bytes = 8,000,000,000 bytes = <b>8 GB</b></span></p>
148-
149-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
150-
151-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>2. Using
152-
float16 (2 bytes per parameter): </span></p>
153-
154-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>2,000,000,000
155-
× 2 bytes = 4,000,000,000 bytes = <b>4 GB</b></span></p>
156-
157-
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
141+
<p class=MsoNormal><img border=0 width=1099 height=239
142+
src="doc_files/image001.png"></p>
158143

159144
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>In order to
160145
run bigger models on our machines, we use a solution called <b>quantization</b>,
@@ -192,25 +177,15 @@
192177
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>After
193178
quantizing a 2-billion-parameter model (using integers):</span></p>
194179

195-
<p class=MsoNormal><b><span style='font-size:14.0pt;line-height:115%'>Using
196-
int8 (1 byte):</span></b></p>
197-
198-
<ul style='margin-top:0in' type=disc>
199-
<li class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>2B × 1
200-
byte = <b>2 GB</b></span></li>
201-
</ul>
202-
203-
<p class=MsoNormal><b><span style='font-size:14.0pt;line-height:115%'>Using
204-
int4 (0.5 byte):</span></b></p>
180+
<p class=MsoNormal><img border=0 width=928 height=180
181+
src="doc_files/image002.png"></p>
205182

206183
<ul style='margin-top:0in' type=disc>
207-
<li class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>2B × 0.5
208-
byte = <b>1 GB</b></span></li>
209184
<li class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></li>
210185
</ul>
211186

212187
<p class=MsoNormal><img border=0 width=1467 height=744
213-
src="doc_files/image001.jpg"></p>
188+
src="doc_files/image003.jpg"></p>
214189

215190
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
216191

@@ -227,7 +202,7 @@
227202
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
228203

229204
<p class=MsoNormal><img border=0 width=1465 height=655
230-
src="doc_files/image002.jpg"></p>
205+
src="doc_files/image004.jpg"></p>
231206

232207
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 2</span></p>
233208

@@ -290,7 +265,7 @@
290265
locally.</span></p>
291266
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
292267
<p class=MsoNormal><img border=0 width=1410 height=553
293-
src="doc_files/image003.jpg"></p>
268+
src="doc_files/image005.jpg"></p>
294269
</td>
295270
<td style='padding:.75pt .75pt .75pt .75pt'>
296271
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -310,7 +285,7 @@
310285
versions that we can run locally.</span></p>
311286
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
312287
<p class=MsoNormal><img border=0 width=1304 height=788
313-
src="doc_files/image004.png"></p>
288+
src="doc_files/image006.png"></p>
314289
</td>
315290
<td style='padding:.75pt .75pt .75pt .75pt'>
316291
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -327,7 +302,7 @@
327302
have the same number of parameters. </span></p>
328303
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
329304
<p class=MsoNormal><img border=0 width=1360 height=562
330-
src="doc_files/image005.jpg"></p>
305+
src="doc_files/image007.jpg"></p>
331306
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
332307
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 5</span></p>
333308
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Originally,
@@ -338,7 +313,7 @@
338313
associated with quantized models.</span></p>
339314
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>We should
340315
have a GPU capable of running the model, along with enough VRAM to load the
341-
quantized parameters. However, if we don’t have a suitable GPU, that's okay</span></p>
316+
quantized parameters. However, if we don't have a suitable GPU, that's okay</span></p>
342317
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>in this
343318
case, the CPU and regular system RAM can also handle the model, though it
344319
will run more slowly.</span></p>
@@ -348,7 +323,7 @@
348323
hardware profile.</span></p>
349324
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
350325
<p class=MsoNormal><img border=0 width=1400 height=681
351-
src="doc_files/image006.jpg"></p>
326+
src="doc_files/image008.jpg"></p>
352327
</td>
353328
<td style='padding:.75pt .75pt .75pt .75pt'>
354329
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -364,7 +339,7 @@
364339
Settings.</span></p>
365340
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
366341
<p class=MsoNormal><img border=0 width=1496 height=797
367-
src="doc_files/image007.jpg"></p>
342+
src="doc_files/image009.jpg"></p>
368343
</td>
369344
<td style='padding:.75pt .75pt .75pt .75pt'>
370345
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -396,7 +371,7 @@
396371
end.</span></p>
397372
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
398373
<p class=MsoNormal><img border=0 width=1484 height=618
399-
src="doc_files/image008.jpg"></p>
374+
src="doc_files/image010.jpg"></p>
400375
</td>
401376
<td style='padding:.75pt .75pt .75pt .75pt'>
402377
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -412,7 +387,7 @@
412387
back to the quantized model, you'll see the hardware profile I shared.</span></p>
413388
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
414389
<p class=MsoNormal><img border=0 width=1409 height=442
415-
src="doc_files/image009.jpg"></p>
390+
src="doc_files/image011.jpg"></p>
416391
</td>
417392
<td style='padding:.75pt .75pt .75pt .75pt'>
418393
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
@@ -442,7 +417,7 @@
442417
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
443418

444419
<p class=MsoNormal><img border=0 width=1416 height=562
445-
src="doc_files/image010.jpg"></p>
420+
src="doc_files/image012.jpg"></p>
446421

447422
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 10</span></p>
448423

@@ -454,7 +429,7 @@
454429
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
455430

456431
<p class=MsoNormal><img border=0 width=1434 height=595
457-
src="doc_files/image011.jpg"></p>
432+
src="doc_files/image013.jpg"></p>
458433

459434
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 11</span></p>
460435

@@ -465,7 +440,7 @@
465440
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
466441

467442
<p class=MsoNormal><img border=0 width=1402 height=621 id="Picture 1"
468-
src="doc_files/image012.jpg"></p>
443+
src="doc_files/image014.jpg"></p>
469444

470445
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 12</span></p>
471446

@@ -481,7 +456,7 @@
481456
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
482457

483458
<p class=MsoNormal><img border=0 width=1275 height=696 id="Picture 2"
484-
src="doc_files/image013.jpg"></p>
459+
src="doc_files/image015.jpg"></p>
485460

486461
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 13</span></p>
487462

@@ -496,7 +471,7 @@
496471
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
497472

498473
<p class=MsoNormal><img border=0 width=1327 height=606 id="Picture 3"
499-
src="doc_files/image014.jpg"></p>
474+
src="doc_files/image016.jpg"></p>
500475

501476
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 14</span></p>
502477

@@ -510,7 +485,7 @@
510485
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
511486

512487
<p class=MsoNormal><img border=0 width=1284 height=501 id="Picture 4"
513-
src="doc_files/image015.png"></p>
488+
src="doc_files/image017.png"></p>
514489

515490
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 15</span></p>
516491

@@ -521,7 +496,7 @@
521496
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
522497

523498
<p class=MsoNormal><img border=0 width=1249 height=806 id="Picture 5"
524-
src="doc_files/image016.png"></p>
499+
src="doc_files/image018.png"></p>
525500

526501
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 16</span></p>
527502

@@ -531,7 +506,7 @@
531506
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
532507

533508
<p class=MsoNormal><img border=0 width=1271 height=658 id="Picture 6"
534-
src="doc_files/image017.jpg"></p>
509+
src="doc_files/image019.jpg"></p>
535510

536511
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 17</span></p>
537512

@@ -541,7 +516,7 @@
541516
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
542517

543518
<p class=MsoNormal><img border=0 width=1325 height=771 id="Picture 7"
544-
src="doc_files/image018.png"></p>
519+
src="doc_files/image020.png"></p>
545520

546521
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 18</span></p>
547522

@@ -553,7 +528,7 @@
553528
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
554529

555530
<p class=MsoNormal><img border=0 width=1186 height=798 id="Picture 8"
556-
src="doc_files/image019.png"></p>
531+
src="doc_files/image021.png"></p>
557532

558533
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 19</span></p>
559534

@@ -573,7 +548,7 @@
573548
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
574549

575550
<p class=MsoNormal><img border=0 width=1371 height=688 id="Picture 9"
576-
src="doc_files/image020.jpg"></p>
551+
src="doc_files/image022.jpg"></p>
577552

578553
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 20</span></p>
579554

@@ -588,7 +563,7 @@
588563
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
589564

590565
<p class=MsoNormal><img border=0 width=1380 height=643 id="Picture 10"
591-
src="doc_files/image021.jpg"></p>
566+
src="doc_files/image023.jpg"></p>
592567

593568
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 21</span></p>
594569

@@ -604,7 +579,7 @@
604579
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
605580

606581
<p class=MsoNormal><img border=0 width=1407 height=642 id="Picture 11"
607-
src="doc_files/image022.jpg"></p>
582+
src="doc_files/image024.jpg"></p>
608583

609584
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 22</span></p>
610585

@@ -615,7 +590,7 @@
615590
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
616591

617592
<p class=MsoNormal><img border=0 width=1444 height=761 id="Picture 12"
618-
src="doc_files/image023.png"></p>
593+
src="doc_files/image025.png"></p>
619594

620595
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 23</span></p>
621596

@@ -630,7 +605,7 @@
630605
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
631606

632607
<p class=MsoNormal><img border=0 width=1452 height=596 id="Picture 13"
633-
src="doc_files/image024.jpg"></p>
608+
src="doc_files/image026.jpg"></p>
634609

635610
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 24</span></p>
636611

@@ -651,7 +626,7 @@
651626
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
652627

653628
<p class=MsoNormal><img border=0 width=1303 height=796 id="Picture 14"
654-
src="doc_files/image025.png"></p>
629+
src="doc_files/image027.png"></p>
655630

656631
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 25</span></p>
657632

@@ -661,7 +636,7 @@
661636
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
662637

663638
<p class=MsoNormal><img border=0 width=1385 height=736 id="Picture 15"
664-
src="doc_files/image026.png"></p>
639+
src="doc_files/image028.png"></p>
665640

666641
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 26</span></p>
667642

@@ -672,7 +647,7 @@
672647
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
673648

674649
<p class=MsoNormal><img border=0 width=1423 height=667 id="Picture 16"
675-
src="doc_files/image027.png"></p>
650+
src="doc_files/image029.png"></p>
676651

677652
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 27</span></p>
678653

@@ -682,7 +657,7 @@
682657
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>&nbsp;</span></p>
683658

684659
<p class=MsoNormal><img border=0 width=1237 height=836 id="Picture 17"
685-
src="doc_files/image028.png"></p>
660+
src="doc_files/image030.png"></p>
686661

687662
<p class=MsoNormal><span style='font-size:14.0pt;line-height:115%'>Figure 28</span></p>
688663

‎docs/doc_files/image003.jpg‎

28.6 KB
Loading

‎docs/doc_files/image005.jpg‎

9.38 KB
Loading

‎docs/doc_files/image006.png‎

272 KB
Loading

‎docs/doc_files/image007.jpg‎

-17 KB
Loading

‎docs/doc_files/image008.jpg‎

4.79 KB
Loading

‎docs/doc_files/image009.jpg‎

25.8 KB
Loading

‎docs/doc_files/image010.jpg‎

2.43 KB
Loading

‎docs/doc_files/image011.jpg‎

-16.4 KB
Loading

‎docs/doc_files/image012.jpg‎

18 KB
Loading

0 commit comments

Comments
 (0)