Stan performance tests fail when setting macro NDEBUG

[SteveBronder]

Description

Scrounging for optimizations wherever I can I noticed we did not set the NDEBUG macro flag which can have some nice teny benefits. However setting the NDEBUG macro at compile time causes the SIR test to fail. If we don't support NDEBUG being defined we should catch it and throw an error message. Otherwise we should figure out how that happens. @wds15 have you ever seen test failures for that before?

Example

git clone --recursive https://github.com/stan-dev/performance-tests-cmdstan
cd performance-tests-cmdstan

replace compare-git-hashes.py with the version here that changes the compiler flags on the second iteration.

./compare-git-hashes.sh "stat_comp_benchmarks/benchmarks/sir/ -j32 --runs 1" develop develop develop develop

Failure's are here everything is off by a good bit.

Expected Output

Tests pass

Current Version:

v3.2.0

[wds15]

This is new to me and should be resolved.

[syclik]

@SteveBronder, I just ran it and it didn't reproduce.

Mind making it a little simpler for me to follow along? Can you commit the changes on a branch and I'll see if I can reproduce it locally?

[SteveBronder]

I'll retry this tmrw. Would be great if this went away magically. Dan what compiler and cpu are you using?

[syclik]

I'm on a MacBook Pro: $ clang++ --version Apple clang version 12.0.0 (clang-1200.0.26.2) Target: x86_64-apple-darwin19.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin 2.9 GHz Quad-Core Intel Core i7

…

On Wed, Nov 25, 2020 at 12:45 AM Steve Bronder ***@***.***> wrote: I'll retry this tmrw. Would be great if this went away magically. Dan what compiler and cpu are you using? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#1939 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADH6F5JXTS2BFA6CEYRPDDSRSKXTANCNFSM4N5UBEOA> .

[syclik]

@SteveBronder, any updates?