Skip to content

"thin" argument affects diagnostics #1185

Open
@ksvanhorn

Description

@ksvanhorn

Describe the bug
If I use the same seed (to ensure reproducibility) and do two model$fit() runs with identical arguments except that one has thin=4, I get fewer divergent transitions, fewer max_treedepth transitions, and worse EBFMI reported for the run that has thinning. (Probably an underlying Stan issue, not just cmdstanr)

To Reproduce

model <- cmdstanr::cmdstan_model(path_to_funnel_model)

fit1 <- model$sample(data=list(), seed=12345678, chains=1, iter_warmup=1000, iter_sampling=4000)
fit1$diagnostic_summary()
# ^^^ num_divergent = 4, num_max_treedepth = 2, ebfmi = 0.08203262

fit2 <- model$sample(data=list(), seed=12345678, chains=1, iter_warmup=1000, iter_sampling=4000, thin=4)
fit2$diagnostic_summary()
# ^^^ num_divergent = 0, num_max_treedepth = 1, ebfmi = 0.2993997

Expected behavior
Thinning should only affect the number of draws saved, not the diagnostics. A divergent transition is an indication of a possible problem whether or not you save it.

Operating system
Your operating system (e.g. mac os x 10.15, windows 10, etc.)

CmdStanR version number
Your CmdStanR version number (e.g. from packageVersion("cmdstanr")).

Additional context
cmdstanr version 0.6.0.9000
R version 4.2.2
macOS 13.4 (Ventura)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions