Open
Description
It can happen that the difference in the confidence intervals is larger than the 0.005
you are expecting:
library(fwildclusterboot)
set.seed(123123)
dqrng::dqRNGkind("Xoroshiro128+")
dqrng::dqset.seed(123123)
data1 <<- fwildclusterboot:::create_data(
N = 1000,
N_G1 = 20,
icc1 = 0.5,
N_G2 = 20,
icc2 = 0.2,
numb_fe1 = 10,
numb_fe2 = 10,
#seed = 908361239,
seed = 123123,
weights = 1:N / N
)
lm_fit <- lm(proposition_vote ~ treatment + log_income,
data = data1
)
boot1 <- boottest(lm_fit,
param = c("log_income"),
clustid = c("group_id2"),
B = 99999,
sampling = "dqrng"
)
#> Warning: Please note that the seeding behavior for random number generation for
#> `boottest()` has changed with `fwildclusterboot` version 0.13.
#>
#> It will no longer be possible to exactly reproduce results produced by versions
#> lower than 0.13.
#>
#> If your prior results were produced under sufficiently many bootstrap
#> iterations, none of your conclusions will change. For more details about this
#> change, please read the notes in
#> [news.md](https://cran.r-project.org/web/packages/fwildclusterboot/news/news.html).
#> This warning is displayed once per session.
#> Too guarantee reproducibility, don't forget to set a global random seed
#> **both** via `set.seed()` and `dqrng::dqset.seed()`.
#> This message is displayed once every 8 hours.
boot2 <- boottest(lm_fit,
param = c("log_income"),
clustid = c("group_id2"),
B = 99999,
sampling = "standard"
)
all.equal(confint(boot1), confint(boot2))
#> [1] "Mean relative difference: 0.005275242"
Created on 2024-05-13 with reprex v2.1.0
Running the the two boottest()
repeatedly w/o fixing the seed I see differences up to 0.007
. This probably explains the surprising results I got in daqana/dqrng#80 when doing reverse dependency checks for the new version of dqrng
.
Could you upload a version of fwildclusterboot
that only changes the the expectation w.r.t. the confidence interval?
expect_equal(confint(boot1), confint(boot2), tolerance = 0.0075)
Note: I am basing my analysis on v0.13, which I think corresponds to the current CRAN version.
Metadata
Metadata
Assignees
Labels
No labels