Re: [Gretl-users] Principal Components Analysis Misbehavior
On Mon, 17 Dec 2012, Riccardo (Jack) Lucchetti wrote:
On Mon, 17 Dec 2012, Henrique Andrade wrote:
> Dear Allin, I am sorry for insist with this subject, but I think the
> changes you made did not fix (completely) the misbehavior.
> Please take a look at the following Hansl code:
>
> <hansl>
> open australia.gdt --quiet
>
> loop i=1..210 --quiet
> pca PAU PUS E --save[1] --quiet
> pca E IAU IUS --save[1] --quiet
> endloop
> </hansl>
>
> After 208 repetitions all new variables are saved with the same
> name: "PC1zzzzzzzzzzzzzzzz"
>
> In my humble opinion, we could handle this in two ways:
>
> 1. Make the sequence of new series as PC1, PC1a, ..., PC1z,
> PC1aa, PC1ab, ..., PC1az, PC1ba, ...
>
> 2. Or make the new series overwrite the old ones.
It seems to me that the only long-term fix to this issue would be endowing
the pca command with a --facname option, to control the root of the generated
factors. So you could have something like the following
<hansl>
open australia.gdt --quiet
loop i=1..210 --quiet
sprintf fn1 "boo_%d" i
sprintf fn2 "zaz_%d" i
pca PAU PUS E --save[1] --quiet --facname=@fn1
pca E IAU IUS --save[1] --quiet --facname=@fn2
endloop
</hansl>
The above may be useful in other contexts too.
Agreed. However, Henrique's example exposed a logic bug in the
code that was supposed to make varnames unique. This is now
fixed in CVS, in the sense that we support up to 999999
"unique-ified" instances of a name such as PC1, which is
better than 208 ;-)
While I was at it, I fixed another two bugs. The fixes raise
backward compatibility issues, but I think they have to be
made.
1) Gretl was accepting the broken syntax "--save[1]" for the
save option to pca. The correct syntax (see help pca) is,
e.g., "--save=1".
The reason the syntax error was getting through is that
gretl's option parser was doing something either not very
clever or too clever, depending on how you look at it. That
is, we'd accept a "too long" form of an option, provided it
started with a valid option string. The original intent behind
this was that the user might (e.g.) not remember whether the
stacked cross-section option to "setobs" was
"--stacked-cross-section" (it is) or
"--stacked-cross-sections" (it's not)
so we'd be "helpful" by accepting the latter as well. But this
allowed through Henrique's trailing "[1]", which should not
happen since it's masking a definite bug in the script.
2) While thinking about that I also got to thinking about
gretl's acceptance of _abbreviated_ versions of option
strings. In principle that's fine (unlike the case above),
except that we really shouldn't accept ambiguous
abbreviations, -- as in "--stacked" (which could mean
--stacked-cross-sectiion or --stacked-time-series). Up till
now we've been mapping an abbreviation to the first match we
find. Now we reject the abbreviation with an error message if
it matches more than one full option string.
Allin Cottrell