Skip to content

Stan using the same seed=1234 | strange performance tests #3

Open
@serban-nicusor-toptal

Description

@serban-nicusor-toptal
method = sample (Default)
  sample
    num_samples = 1000 (Default)
    num_warmup = 1000 (Default)
    save_warmup = 0 (Default)
    thin = 1 (Default)
    adapt
      engaged = 1 (Default)
      gamma = 0.050000000000000003 (Default)
      delta = 0.80000000000000004 (Default)
      kappa = 0.75 (Default)
      t0 = 10 (Default)
      init_buffer = 75 (Default)
      term_buffer = 50 (Default)
      window = 25 (Default)
    algorithm = hmc (Default)
      hmc
        engine = nuts (Default)
          nuts
            max_depth = 10 (Default)
        metric = diag_e (Default)
        metric_file =  (Default)
        stepsize = 1 (Default)
        stepsize_jitter = 0 (Default)
id = 0 (Default)
data
  file = XXX
init = 2 (Default)
random
  seed = 1234
output
  file = XXX
  diagnostic_file =  (Default)
  refresh = 100 (Default)

Tests I found during Jenkins runs that give strange performance reports on the same machine in subsequent runs.

PrevBuildURI90% | CurrentBuildURI90% | RelativeDiff | RelativeDiff(%)

286               40526               40240           14069.93%
example-models/bugs_examples/vol1/pump/pump.data.R
---
443               42348               41905            9459.37%
example-models.bugs_examples.vol1.dyes.dyes  
---
23237                 533              -22704             -97.71%
example-models.bugs_examples.vol1.equiv.equiv  
---
8302               39174               30872             371.86%
example-models.bugs_examples.vol1.leuk.leuk 
---
7807                2274               -5533             -70.87%
example-models.bugs_examples.vol1.mice.mice 
---
163                 365                 202             123.93%
example-models.bugs_examples.vol2.beetles.beetles_cloglog   
---
7807               39129               31322             401.20%
example-models.bugs_examples.vol1.mice.mice
---
23237                 506              -22731             -97.82%
example-models.bugs_examples.vol1.equiv.equiv
---
 153              114870              114717           74978.43%
example-models.bugs_examples.vol2.beetles.beetles_probit

Activity

changed the title [-]Stan using the same seed=1234[/-] [+]Stan using the same seed=1234 | strange performance tests[/+] on Apr 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @seantalts@serban-nicusor-toptal

      Issue actions

        Stan using the same seed=1234 | strange performance tests · Issue #3 · stan-dev/performance-tests-cmdstan