Skip to content

Commit 4fcdeb2

Browse files
committed
Adding more information for Table 2.
1 parent 3d843b8 commit 4fcdeb2

File tree

2 files changed

+7
-5
lines changed

2 files changed

+7
-5
lines changed

README.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -261,7 +261,7 @@ Several of our experiments (the experiments which produce results for the tables
261261

262262
#### Table 1
263263

264-
For Table 1, you'll see a print out which looks like the following (specific numbers will depend on your runtime environment):
264+
For Table 1, you'll see a print out which looks like the following (specific numbers may depend on your runtime environment):
265265

266266
```
267267
GenJAX VI timings:
@@ -278,7 +278,7 @@ The "columns" of the print out match to the batch size: first array returned by
278278

279279
#### Table 2
280280

281-
To generate Table 2 in the paper, we use [`pytest-benchmark`](https://pypi.org/project/pytest-benchmark/) to generate timing statistics. The print out will likely look something like this (the specific numbers will depend on your runtime environment -- but the general trend, that our timings are orders of magnitude faster than Pyro, doesn't depend on runtime):
281+
To generate Table 2 in the paper, we use [`pytest-benchmark`](https://pypi.org/project/pytest-benchmark/) to generate timing statistics. The print out will likely look something like this (the specific numbers may depend on your runtime environment -- but the general trend, that our timings are orders of magnitude faster than Pyro, doesn't depend on runtime):
282282

283283
![table_2_print_out](table_2_print_out.png)
284284

@@ -297,7 +297,7 @@ Each of the names on the right hand side of the arrows above correspond to parti
297297

298298
#### Table 4
299299

300-
For Table 4, (as mentioned in **Abbreviations**), you'll see a print out which looks like this:
300+
For Table 4, (as mentioned in **Abbreviations**), you'll see a print out which looks like this (specific numbers may depend on your runtime environment):
301301
```
302302
poetry run python experiments/table_4_objective_values/genjax_cone.py
303303
ELBO:
@@ -330,10 +330,12 @@ Pyro IWAE(K = 5):
330330
(tensor(7.8314), tensor(2.4545))
331331
```
332332

333-
We've added spaces in the above print out between the independent experiments involved in Table 4.
333+
We've added spaces in the above print out between the independent experiments involved in producing Table 4:
334+
335+
![table_4](table_4.png)
334336

335337
A few things to note:
336-
* (**Abbreviations**) The IWAE label is equivalent to IWELBO, as is the RenyiELBO name (from the experiments on Pyro and NumPyro).
338+
* (**Abbreviations and name changes**) In the print out, the IWAE and RenyiELBO labels are equivalent to IWELBO in Table 4. The number of importance weighted samples (in the print out, `K = 5`) is converted to $n = 5$ in Table 4 (e.g. for `Pyro IWAE(K = 5)` and our `IWAE(K = 5)`). In the experiments, `HVI-ELBO` is equivalent to `HVI` in Table 4. All other names are the same between the print out and Table 4.
337339
* All system comparison experiments (Pyro and NumPyro) are labeled with their names in this table.
338340
* (**What the numbers mean**) We did not report the standard deviation in this table: for each experiment, the first array is the mean over several trials, and the second is standard deviation.
339341
* **(Objective value convention)** Pyro and NumPyro use the negative ELBO as their objective (negative ELBO minimization), to compare with `genjax.vi` (and what we've done in Table 4) is apply a minus sign to the Pyro and NumPyro results.

table_4.png

46.4 KB
Loading

0 commit comments

Comments
 (0)