More stable implementation of neg_binomial_2_log_lpmf

[martinmodrak]

Summary

These changes mirror those done for neg_binomial_2_lpmf in #1497, binomial_coefficient_log is used to avoid the necessity to delegate to Poisson.

Tests

• Precomputed values and gradients in Mathematice
• Derivatives via complex step
• Matches neg_binomial_2_lpmf(n, exp(eta), phi)
• Tests of values and derivatives against analytical solutions for n = 0 and n = 1

Side Effects

The way the density is computed changed to improve stability, at the cost of tangling eta and phi together earlier, so less computation can be avoided when propto=true and phi is fixed.

Release notes

More stable implementation of neg_binomial_2_log_lpmf

Checklist

• Math issue neg_binomial_2_log_lpmf is not stable for large phi #1495

• Copyright holder: Institute of Microbiology of the Czech Academy of Sciences

By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
- Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
- Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

• the basic tests are passing

  • unit tests pass (to run, use: ./runTests.py test/unit)
  • header checks pass, (make test-headers)
  • dependencies checks pass, (make test-math-dependencies)
  • docs build, (make doxygen)
  • code passes the built in C++ standards checks (make cpplint)

• the code is written in idiomatic C++ and changes are documented in the doxygen

• the new changes are tested

[martinmodrak]

Ready for review. Maybe @bbbales2 could take care of it? Hopefully we can get this in for the 2.23 release... Thanks!

[rok-cesnovar]

I am sure we can get it in. Note that this is a bugfix so the deadline is actually 20th of April. But if we can get it for the release candidate that would be even better.

[stan-buildbot]

---

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.79	4.91	0.98	-2.44% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	0.97	-2.65% slower
eight_schools/eight_schools.stan	0.09	0.09	1.03	2.85% faster
gp_regr/gp_regr.stan	0.22	0.22	1.01	0.69% faster
irt_2pl/irt_2pl.stan	6.44	6.49	0.99	-0.76% slower
performance.compilation	89.57	87.21	1.03	2.64% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.53	7.52	1.0	0.12% faster
pkpd/one_comp_mm_elim_abs.stan	21.0	21.44	0.98	-2.07% slower
sir/sir.stan	90.91	95.01	0.96	-4.51% slower
gp_regr/gen_gp_data.stan	0.05	0.05	1.0	0.34% faster
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.95	2.95	1.0	0.05% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.31	0.3	1.02	1.97% faster
arK/arK.stan	1.74	1.74	1.0	0.12% faster
arma/arma.stan	0.66	0.65	1.0	0.49% faster
garch/garch.stan	0.52	0.51	1.02	1.66% faster
Mean result: 0.999391039988

---

Jenkins Console Log
Blue Ocean
Commit hash: 766de89

---

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

[bbbales2]

@serban-nicusor-toptal Looks like this test is popping up in downstream performance:

Cannot contact old-imac: java.lang.InterruptedException
make: mkdir: Exec format error
make -j6 test/performance/logistic_test failed
exit now (04/11/20 09:29:58 EDT)

Does that mean a computer crashed? Anything we can do to un-crash that haha?

@martinmodrak this looks good to me. I'll merge after we get the tests figured out.

[serban-nicusor-toptal]

Hey, reduced the parallel variable from 6 to 4 so we rule out that the load may crash the machine. Gonna restart the build.

[stan-buildbot]

---

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.8	4.86	0.99	-1.23% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	0.97	-2.88% slower
eight_schools/eight_schools.stan	0.09	0.09	0.99	-0.98% slower
gp_regr/gp_regr.stan	0.22	0.22	1.01	0.72% faster
irt_2pl/irt_2pl.stan	6.43	6.45	1.0	-0.25% slower
performance.compilation	88.36	87.31	1.01	1.18% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.58	7.58	1.0	0.03% faster
pkpd/one_comp_mm_elim_abs.stan	21.88	20.02	1.09	8.47% faster
sir/sir.stan	92.82	92.9	1.0	-0.09% slower
gp_regr/gen_gp_data.stan	0.05	0.05	1.0	0.43% faster
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.95	2.97	1.0	-0.44% slower
pkpd/sim_one_comp_mm_elim_abs.stan	0.3	0.32	0.94	-6.72% slower
arK/arK.stan	1.74	1.74	1.0	-0.05% slower
arma/arma.stan	0.66	0.66	1.0	-0.42% slower
garch/garch.stan	0.51	0.51	1.0	0.09% faster
Mean result: 0.999458741879

---

Jenkins Console Log
Blue Ocean
Commit hash: 766de89

---

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

[bbbales2]

@serban-nicusor-toptal looks like it failed again. Ideas?

[serban-nicusor-toptal]

There is a known but for Jenkins with github, builds that are replayed won't show up here ( in the bottom continous-integration ).
The build looks like running for now.
To be sure you can always go to the PR page and see what's the latest build here

[stan-buildbot]

---

Name	Old Result	New Result	Ratio	Performance change( 1 - new / old )
gp_pois_regr/gp_pois_regr.stan	4.87	4.92	0.99	-0.95% slower
low_dim_corr_gauss/low_dim_corr_gauss.stan	0.02	0.02	0.99	-1.2% slower
eight_schools/eight_schools.stan	0.09	0.09	1.05	5.03% faster
gp_regr/gp_regr.stan	0.22	0.22	0.99	-1.41% slower
irt_2pl/irt_2pl.stan	6.5	6.46	1.01	0.64% faster
performance.compilation	88.26	87.13	1.01	1.27% faster
low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan	7.52	7.52	1.0	0.02% faster
pkpd/one_comp_mm_elim_abs.stan	20.58	19.89	1.03	3.36% faster
sir/sir.stan	94.47	96.41	0.98	-2.05% slower
gp_regr/gen_gp_data.stan	0.05	0.05	0.97	-2.59% slower
low_dim_gauss_mix/low_dim_gauss_mix.stan	2.96	2.95	1.0	0.11% faster
pkpd/sim_one_comp_mm_elim_abs.stan	0.3	0.31	0.97	-2.74% slower
arK/arK.stan	1.75	1.73	1.01	1.14% faster
arma/arma.stan	0.66	0.66	0.99	-0.68% slower
garch/garch.stan	0.51	0.51	1.0	0.36% faster
Mean result: 1.00064272311

---

Jenkins Console Log
Blue Ocean
Commit hash: 766de89

---

Machine information

ProductName: Mac OS X ProductVersion: 10.11.6 BuildVersion: 15G22010

CPU:
Intel(R) Xeon(R) CPU E5-1680 v2 @ 3.00GHz

G++:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

Clang:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin15.6.0
Thread model: posix

[bbbales2]

Thanks @martinmodrak, it's in! And thanks for the help @serban-nicusor-toptal!