4:30 p.m.
Date: Fri, 16 Apr 2010 18:14:08 -0400 (EDT)
From: Allin Cottrell <cottrell(a)wfu.edu>
Subject: Re: [Gretl-devel] NA and nan: next steps
To: Gretl development <gretl-devel(a)lists.wfu.edu>
Message-ID: <Pine.A41.4.58.1004161812050.2326900(a)f1n11.sp2net.wfu.edu>
Content-Type: TEXT/PLAIN; charset=US-ASCII
On Sat, 17 Apr 2010, Sven Schreiber wrote:
Allin Cottrell schrieb:
> On Wed, 14 Apr 2010, Gordon Hughes wrote:
>
>> Can I raise a dissenting voice? Do you REALLY want to expend the
>> effort to distinguishing between NA and NaN in every single procedure
>> and (presumably) every function, etc? It would be even worse if you
>> added +/-Inf. My reaction is that there are better ways to spend
>> time in developing the program.
>
> I have no desire to spend a lot of time in this area. I suspect
> there's an intractable problem here, which has to be resolved by
> fiat. In principle, NA and NaN are different things, which is
> particularly apparent in the case of evaluating 0*NA versus 0*NaN.
>
> The statistical programs that we've had reports on to date on this
> list resolve the issue by treating NAs as if they were NaNs;
I don't mean to suggest any implication for gretl's development here,
but it seems to me that this statement is not correct as regards Octave
and R; at least from what quick googling revealed to me, since I'm not
an expert in either of those packages. Both Octave (/Matlab) and R seem
to distinguish NA and NaN (and I guess even +-Inf) AFAICS.
->OK, they may make _some_ distinction, but if they evaluate 0*NA as-
->NA (as we've heard) then they are not doing it right.
->Allin
Except if 0 is a dummy variable. In that case 0*NA = NA., 0 is not really
zero in ordinal data--it's arbitrary and just indicates a category.
Otherwise recoding the dummy variables changes the effective data set and
lead to unexpected results. If 0 is really zero in the cardinal sense then
gretl handles this correctly and R/Octave do not. If 0 is a categorical
variable gretl gets it wrong and Octave/R get it right. One solution would
be to declare dummy variables as factors (like Jeff Racine does in his
semiparametric models) so that the handling of the interactions could be
right in both instances.
I like what gretl does for cardinal data. But I'm not sure this is what I
would want for ordinal/categorical data.
Lee
4:55 p.m.
On Sat, 17 Apr 2010, Lee Adkins wrote:
->OK, they may make _some_ distinction, but if they evaluate 0*NA
as-
->NA (as we've heard) then they are not doing it right.
Except if 0 is a dummy variable. In that case 0*NA = NA., 0 is
not really zero in ordinal data--it's arbitrary and just
indicates a category. Otherwise recoding the dummy variables
changes the effective data set and lead to unexpected results.
Good point. Just thinking it through: If one had (D1: female = 1,
male = 0) and (D2: married = 1, unmarried = 0), and for some
individuals D2 was NA, and then one created an interaction D1*D2,
one would wrongly categorize males of unknown marital status as
unmarried males, if 0*NA = 0. I'll have to think about that.
Allin
9:43 p.m.
Allin Cottrell schrieb:
On Sat, 17 Apr 2010, Lee Adkins wrote:
> ->OK, they may make _some_ distinction, but if they evaluate 0*NA as-
> ->NA (as we've heard) then they are not doing it right.
>
> Except if 0 is a dummy variable. In that case 0*NA = NA., 0 is
> not really zero in ordinal data--it's arbitrary and just
> indicates a category. Otherwise recoding the dummy variables
> changes the effective data set and lead to unexpected results.
Good point. Just thinking it through: If one had (D1: female = 1,
male = 0) and (D2: married = 1, unmarried = 0), and for some
individuals D2 was NA, and then one created an interaction D1*D2,
one would wrongly categorize males of unknown marital status as
unmarried males, if 0*NA = 0. I'll have to think about that.
Yep, this whole thread started when I was handling dummies and got
unexpected results like that.
Now Lee points out that gretl assumes that NAs are (unknown) numbers on
a cardinal scale. But if you allow me to rephrase myself a little (and
without referring to NaN's this time, which is a related but slightly
different issue):
I am actually also worried about assuming _any_ type of number for NAs.
What does the abbrev. "NA" stand for in English? I can think of at least
two meanings: "not available" or "not applicable". The interpretation
of
"not available" may well allow to treat NAs as numbers, but I really
think that "not applicable" will usually not be hiding a meaningful
number underneath. And it seems to me that missing values in
economic/social statistics quite often carry the meaning of "not
applicable". At least they did in my unbalanced panel data set where I
encountered the problems.
All in all, while the cleanest solution may be to introduce and properly
differentiate NAs, NaNs, Infs and all the rest, wouldn't the easiest way
simply be to make any operation on NA return NA, including 0*NA=NA?
cheers,
sven
5363
days inactive
5363
days old
2 comments
3 participants
participants (3)
-
Allin Cottrell -
Lee Adkins -
Sven Schreiber