Recover higher performance from re-enabling bounds checking.

[ryan-richt]

Ask

We would like a way to recover the improved speed from the version of Stanc3 prior to #849 . This PR re-introduced array bounds checking for safety and user experience at a significant performance cost.

One proposed solution is to create a "debug/test vs production" Stan compiler flag or otherwise option that (1) by default is turned to the safe and informative debug mode to serve new adopters and (2) can be actively set/changed by those with higher performance needs to get the performance.

Background

Many awesome devs and community managers did great work to establish:

• Array bounds checking and other similar checks can incur a significant performance penalty on models (order of 30%), which is a huge problem for Stan users with performance-critical models
• Eliminating array bounds checking in Stanc3 by default causes many new-adopters to generate seg faults at run time, not know why, and post to the forums, possibly turning them off of Stan and its community.

These checks were eliminated / never implemented in Stanc3. Someone did excellent work to support and add them all into Stanc3, which was originally rejected b/c of well-thought-out-and-executed performance tests that showed a 30% performance hit. After the new user issue surfaced several times, that PR with the safety checks was revived and merged, and someone suggested we add this ticket!

[SteveBronder]

I think what we are talking about is just having -DNDEBUG not perform range checks. Would that work for you? It's assuming the overhead here is just in checking the bounds and not the actual rvalue() function call but I think that's a reasonable assumption

[bbbales2]

@ryan-richt it should be possible to use CXXFLAGS to turn off bounds check for production models with -DSTAN_NO_RANGE_CHECKS: stan-dev/math#2423