1:15 p.m.
Hello gretl users,
yet again one of those tsls-problems of mine...
It seems to me that TSLS doesn't calculate the R^2 correctly. Actually,
it produces exactly the same values as the corresponding OLS estimation.
For example, the three-liner
open denmark
ols IDE 0 IBO
tsls IDE 0 IBO; LRY
produces the following output (excerpt):
ols:
Unkorrigiertes R-Quadrat = 0.644317
Korrigiertes R-Quadrat = 0.637606
tsls:
Unkorrigiertes R-Quadrat = 0.644317
Korrigiertes R-Quadrat = 0.637606
I'm going to file a bug report soon if nobody stops me by saying that it
all makes perfect sense...
thanks,
sven
2:12 p.m.
On Thu, 6 Dec 2007, Sven Schreiber wrote:
Hello gretl users,
yet again one of those tsls-problems of mine...
It seems to me that TSLS doesn't calculate the R^2 correctly. Actually, it
produces exactly the same values as the corresponding OLS estimation. For
example, the three-liner
open denmark
ols IDE 0 IBO
tsls IDE 0 IBO; LRY
Hold on a second. Never mind the R^2 index; the tsls command should never
work here (2 regressors and 1 instrument). The fact that you manage to
get an output at all is very very wrong. I'll try to see what goes wrong
as soon as I get a chance (unlikely before tomorrow).
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
5:16 p.m.
On Thu, 6 Dec 2007, Riccardo (Jack) Lucchetti wrote:
On Thu, 6 Dec 2007, Sven Schreiber wrote:
> Hello gretl users,
>
> yet again one of those tsls-problems of mine...
>
> It seems to me that TSLS doesn't calculate the R^2 correctly. Actually, it
> produces exactly the same values as the corresponding OLS estimation. For
> example, the three-liner
>
> open denmark
> ols IDE 0 IBO
> tsls IDE 0 IBO; LRY
Hold on a second. Never mind the R^2 index; the tsls command should never
work here (2 regressors and 1 instrument). The fact that you manage to get an
output at all is very very wrong. I'll try to see what goes wrong as soon as
I get a chance (unlikely before tomorrow).
Ok, folks, nothing to worry about; tsls does work as it should. I was just
unaware of the fact that if the constant is in the regressor list, it gets
automatically added to the instrument list if not already present.
However, there's a "FIXME" comment in that portion of the code.
I think I can see Allin's reasoning here "how can the constant be
endogenous?", but I wonder if this is maybe going a little too far in
protecting the user from his own stupidity. I can't imagine right now a
practical situation when the constant needs to be among the regressors but
not among the instruments, but hey, it's a free world.
So, unless someone steps up in the next couple of days giving me a good
reason for not doing it, I'll remove this feature from CVS (it's easy to
revert the change if needed).
Allin, feel free to put me under trial for high treason when you're back
:-)
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
7:06 p.m.
Riccardo (Jack) Lucchetti schrieb:
Ok, folks, nothing to worry about; tsls does work as it should. I was
just
unaware of the fact that if the constant is in the regressor list, it
gets automatically added to the instrument list if not already present.
...
.
So, unless someone steps up in the next couple of days giving me a
good reason for not doing it, I'll remove this feature from CVS (it's
easy to revert the change if needed).
I share your view about this. However,
it's a backwards-incompatible
change and might break existing scripts. Maybe for some future release
(2.0?) that argument shouldn't count anymore, but until then I'd say
it's a pretty strong point.
On the original issue of R^2, I'll file the bug report soon.
thanks,
sven
8:49 p.m.
Sven Schreiber schrieb:
On the original issue of R^2, I'll file the bug report soon.
"Delay that order"... Actually for my real-world cases it turns out
there isn't anything (obviously) wrong. However, I'm still puzzled by
the 3-liner results I posted earlier. The point estimates are quite
different between ols and tsls -- then how come the correlation between
fitted and observed is the same to five or six digits of precision? Hm.
-sven
1:41 p.m.
On Thu, 6 Dec 2007, Sven Schreiber wrote:
> On the original issue of R^2, I'll file the bug report
soon.
>
"Delay that order"... Actually for my real-world cases it turns
out there isn't anything (obviously) wrong. However, I'm still
puzzled by the 3-liner results I posted earlier. The point
estimates are quite different between ols and tsls -- then how
come the correlation between fitted and observed is the same to
five or six digits of precision? Hm.
"Strange but true". It seems to be in the arithmetic for the case
of one independent variable and one instrument.
nulldata 50
genr x = normal()
genr y = normal()
genr z = normal()
ols y 0 x
ols x 0 z --quiet
genr xhat = $coeff(const) + $coeff(z)*z
ols y 0 xhat
genr yhat = $coeff(const) + $coeff(xhat)*x
R2 = corr(y, yhat)^2
"R2" is numerically identical to the R^2 from the first OLS.
Left as an exercise: prove that this is always the case.
Allin.
1:50 p.m.
Allin Cottrell schrieb:
On Thu, 6 Dec 2007, Sven Schreiber wrote:
>> On the original issue of R^2, I'll file the bug report soon.
>>
>>
> "Delay that order"... Actually for my real-world cases it turns
> out there isn't anything (obviously) wrong. However, I'm still
> puzzled by the 3-liner results I posted earlier. The point
> estimates are quite different between ols and tsls -- then how
> come the correlation between fitted and observed is the same to
> five or six digits of precision? Hm.
>
"Strange but true". It seems to be in the arithmetic for the case
of one independent variable and one instrument.
nulldata 50
genr x = normal()
genr y = normal()
genr z = normal()
ols y 0 x
ols x 0 z --quiet
genr xhat = $coeff(const) + $coeff(z)*z
ols y 0 xhat
genr yhat = $coeff(const) + $coeff(xhat)*x
R2 = corr(y, yhat)^2
"R2" is numerically identical to the R^2 from the first OLS.
Left as an exercise: prove that this is always the case.
Sorry to leave that unclarified, Jack already pointed out the (simple)
solution of this pseudo-mystery directly to me. (Trying to not embarrass
me publicly I guess...) Any linear transformation (or some may call it
affine) of a single variable will leave the linear correlation with
another variable unchanged of course. So the estimated coefficients are
really totally irrelevant in this case.
cheers,
sven
1:02 p.m.
On Thu, 6 Dec 2007, Riccardo (Jack) Lucchetti wrote:
I was just unaware of the fact that if the constant is in the
regressor list, it gets automatically added to the instrument
list if not already present...
I think I can see Allin's reasoning here "how can the constant
be endogenous?", but I wonder if this is maybe going a little
too far in protecting the user from his own stupidity. I can't
imagine right now a practical situation when the constant needs
to be among the regressors but not among the instruments, but
hey, it's a free world.
I don't feel strongly about this, but my first reaction is that
it's a low-cost convenience to the user, to add the constant to
the list of instruments if it's among the regressors (and not
already specified as an instrument). If we're to remove this,
we'd have to check that it doesn't break TSLS in the GUI.
I tend to think that if a user really wants to treat the constant
as endogenous, s/he can go ahead and do the two stages of TSLS
manually.
Allin.
2:08 p.m.
On Mon, 10 Dec 2007, Allin Cottrell wrote:
On Thu, 6 Dec 2007, Riccardo (Jack) Lucchetti wrote:
> I was just unaware of the fact that if the constant is in the
> regressor list, it gets automatically added to the instrument
> list if not already present...
>
> I think I can see Allin's reasoning here "how can the constant
> be endogenous?", but I wonder if this is maybe going a little
> too far in protecting the user from his own stupidity. I can't
> imagine right now a practical situation when the constant needs
> to be among the regressors but not among the instruments, but
> hey, it's a free world.
I don't feel strongly about this, but my first reaction is that
it's a low-cost convenience to the user, to add the constant to
the list of instruments if it's among the regressors (and not
already specified as an instrument). If we're to remove this,
we'd have to check that it doesn't break TSLS in the GUI.
Good point.
I tend to think that if a user really wants to treat the constant
as endogenous, s/he can go ahead and do the two stages of TSLS
manually.
I don't have a very strong opinion on this either, but I think that in a
script it looks weird if you have more regressors than instruments. At
least _my_ first reaction was when I saw Sven's example to think that tsls
was broken.
Sure adding the constant makes sense in those cases, but so does in
various other contexts. For example, when you run a regression in Stata,
the constant is always added automagically unless you specify the
"noconstant" option. We do the same for ARIMAs, so one may conceivably
consider adding the constant by default in all estimation methods,
including ols, for consistency.
The fact remains, that if a user wanted, maybe just to see what happens,
to run a weird tsls model with a constant among the regressors but not
among the instruments, s/he has to do it by hand (which is, I agree, only
a minor inconvenience).
If we decide to keep it in, fine by me, but I'd like to have some warning
message printed out, as per my recent CVS commit (the exact wording could
be softened).
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
5:30 p.m.
The constant is by definition uncorrelated with the residual and
therefore can not be endogenous. I note that the gui always includes
the constant in both the set of regressors and in the instruments
which is generally correct. I like the fact that in FRETL all the
exogenous variables are included in the list of exogenous variables
rather that in stata where one lists all the endogenous and the
exogenous variables not already listed.
Measuring R^2 as 1-RSS/TSS will give a different answer in OLS and in
TSLS which is what one would expect. Is this, perhaps a more
appropriate measure of fit for TSLS. I think that both give the same
answer in ols.
Best Regards
John
On 10/12/2007, Allin Cottrell <cottrell(a)wfu.edu> wrote:
On Thu, 6 Dec 2007, Riccardo (Jack) Lucchetti wrote:
> I was just unaware of the fact that if the constant is in the
> regressor list, it gets automatically added to the instrument
> list if not already present...
>
> I think I can see Allin's reasoning here "how can the constant
> be endogenous?", but I wonder if this is maybe going a little
> too far in protecting the user from his own stupidity. I can't
> imagine right now a practical situation when the constant needs
> to be among the regressors but not among the instruments, but
> hey, it's a free world.
I don't feel strongly about this, but my first reaction is that
it's a low-cost convenience to the user, to add the constant to
the list of instruments if it's among the regressors (and not
already specified as an instrument). If we're to remove this,
we'd have to check that it doesn't break TSLS in the GUI.
I tend to think that if a user really wants to treat the constant
as endogenous, s/he can go ahead and do the two stages of TSLS
manually.
Allin.
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
--
John C Frain
Trinity College Dublin
Dublin 2
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
6:32 p.m.
On Mon, 10 Dec 2007, John C Frain wrote:
The constant is by definition uncorrelated with the residual and
therefore can not be endogenous.
<boring theoretical rant>
The constant cannot by definition be correlated with anything, since it
has no variance. If what you mean is E(u_t * 1) = 0, that is not true by
definition, but by hypothesis. To be more specific: in a linear model
suitable for estimation via tsls it is assumed that E((y - xb)|z) = 0.
Nothing more, nothing less. 99.9999% of the times it's perfectly natural
that z includes the constant, but it doesn't _have_ to.
Perhaps the reason why we're all squabbling over this is a difference in
perception: if one sees tsls as a way to get round the problem of
endogenous regressors, then, well, there's little more to say. I
personally tend to see tsls as nothing but a special case of gmm, not
necessarily tied to a simultaneous system framework, so as far as I'm
concerned the way you specify your orthogonality relationships is no-one
else's business.
</boring theoretical rant>
Really, I don't mind either way. Could we go back to serious work now,
please? :-)
Measuring R^2 as 1-RSS/TSS will give a different answer in OLS and
in
TSLS which is what one would expect. Is this, perhaps a more
appropriate measure of fit for TSLS. I think that both give the same
answer in ols.
The equivalent to R^2 that I personally consider most sensible is the one
proposed by Pesaran & Smith ("A Generalized R^2 Criterion for Regression
Models Estimated by the Instrumental Variables Method", Econometrica, Vol.
62, pp. 705--10.); however, the one we use in gretl now is reasonably
established and I don't personally feel the need to change it.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
6347
days inactive
6351
days old
10 comments
4 participants
participants (4)
-
Allin Cottrell -
John C Frain -
Riccardo (Jack) Lucchetti -
Sven Schreiber