On Fri, 22 Jul 2016, Stefano Fachin wrote:
dear all,
I run into an unexpected behaviour of the "pvalue" utility, which crashes
if required to compute the pvalue of a very small number using a Chi square
with a large number of degrees of freedom. For instance, setting 227
degrees of freedom the pvalue is computed only for numbers larger than
about 0.165. For lower values the reply is "impossible to compute the
pvalue" in the GUI and "NA" in a batch script (v. 2016b, 2016-04-14) Of
course, nobody in his right mind will knowingly want to compute such a
pvalue, but when running in a script it may happen and the NA output does
not look good.
I take your point and we'll see about fixing this.
In this case we calculate the p-value as the complement of the CDF.
At present pvalue(X, 227, 0.2) returns 1.0, which is obviously not
literally correct but is a better approximation to the true value than
is the largest double-precision floating-point value less than 1.0. In
interactive use you'll see a value for the CDF: 1.2e-299. There is no
double-precision representation of 1.0 - 1.2e-299 other than 1.0
itself. So we're OK.
But if you try pvalue(X, 227, 0.1) you get NA. That's because the code
to calculate the CDF fails with numerical underflow. However, it seems
we should tolerate that error and return 1.0 for the p-value (so long
as the argument is non-negative).
It's less clear what we should do in case of underflow if it's the CDF
itself that's requested. Should "too small to calculate" produce NA or
zero?
Allin Cottrell