Re: [Gretl-users] meanc() when matrix has NAs
On Wed, 14 Oct 2015, Allin Cottrell wrote:
On Thu, 15 Oct 2015, Artur T. wrote:
> Dear all,
>
> I would like to compute the mean of a vector which involves NAs, implying
> that the NAs should simply be neglected/not counted. But this is not the
> way gretl handles matrices:
>
> <gretl>
> matrix A = {NA, 1; 2, 2; 1, 3}
> mA = meanc(A)
> mA
> </gretl>
>
> yields
>
> <output>
> mA (1 x 2)
> nan 2
> </output>
>
> Is this intended? For instance, R, it seems, fully neglects the
> NAs and computes the mean.
Yes, it's intended. There's really no such thing as NA in a gretl
matrix, the "closest translation" is NaN (not-a-number) and that's
what gets written into the matrix if you give NA as an element.
And NaN (a standard C-library/IEEE concept) turns everything it
touches into another NaN: x + NaN = NaN, x * NaN = NaN, and so on.
How are NAs (NaNs) getting into your matrix? Why do you think they
ought to be ignored/skipped instead of propagating when doing
calculations on the matrix?
A little more on this. We've been over this issue before (and it's
not an easy one) but I can't find the references right away, so let
me say a few things from "first principles".
When you load data (series) from file, gretl takes NA to represent
genuinely "Not Available" (the data are missing for some reason) and
not "a non-number resulting from an invalid calculation". So when
you apply functions such as mean() and sd() to series, we skip the
NAs when calculating the statistic.
With matrices, however, we want to be able to pass them to
high-performance multi-threaded libraries such as OpenBlas (or just
well proven single-threaded libraries such as Netlib BLAS) and these
third-party libraries have no notion of "missing values" (a notion
inherently foreign to matrix algebra). The nearest equivalent is to
translate NA (= missing) to NaN. But then we have to let these NaNs
behave (propagate) according to the standard IEEE rules, which are
designed to treat NaN as something invalid (as in the log of a
negative number or the quotient 0/0).
So in gretl, if you want NAs to be skipped then work with series,
not matrices; or conversely, allow NAs into your matrices only if
you want them to be treated as NaNs (not "missing" but invalid).
Allin