4:50 p.m.
I was trying to understand how GRETL calculates R^2 for fixed-effects
panel-data models as the value it produces is different from Stata and R's
plm library.
As far as I have been able to understand, the calculation happens in the
function fix_within_stats in the file lib/src/gretl_panel.c as follows
(line 1758-9 in the current CVS head, comment in the original):
/* should we modify R^2 in this way? */
fmod->rsq = 1.0 - (fmod->ess / fmod->tss);
Here fmod->ess is the error-sum-of-squared from the regression on the
demeaned data but fmod->tss has been overwritten earlier in the function
(line 1750) by the TSS from the pooled regression.
I am a newcomer to this list, so apologies if this has already been
discussed, but it is not clear to me whether the above is a useful measure
of goodness of fit, since a high R^2 would be reported even in situations
where the unit dummies explain a lot of the variation in y while the
explicit regressors do not add much.
I would be grateful if someone can point me to some justification in the
literature for this measure. I also think that the way R^2 is calculated
should be discussed in the User's Guide.
Stata's xtreg reports three different measures of "R^2" (see page 10 of
http://www.stata-press.com/manuals/stata12/xt_xtreg.pdf ) none of which
match the one above. R's plm function from the plm library reports an R^2
which equals the "within" R^2 reported by Stata.
Gretl could report the same value simply by not modifying the values in
fmod->rsq.
Regards,
Jyotirmoy Bhattacharya
6:52 p.m.
On Wed, 27 Nov 2013, Jyotirmoy Bhattacharya wrote:
I was trying to understand how GRETL calculates R^2 for
fixed-effects
panel-data models as the value it produces is different from Stata and R's
plm library.
Premise: I'm no big fan of R^2 in any shape or form, so consider my answer
as coming from someone who, in applied work, never even looks at the R^2
index.
However, it may be argued that a measure of the correlation between your
data and the prediction that the model gives you is a desirable
descriptive statistic to have. The problem here is: which model and which
data.
The fixed-effect model could be conceived in two ways which are, in my
mind, equally defensible: (a) a nice, clean way to get rid of the
individual effects by using the fact that in the linear model a sufficient
stsistic is easy to compute or (b) as a clever way to estimate the
"important" parameters of a model in which you want to include (for some
reason) individual dummies. If you take perspective (b), then your data is
the unmodified y and your model includes the unit dummies as well, and
gretl is already doing the "right thing" (especially considering that it
could be useful, for instructional purposes, to show the students that FE
and a regression with lots of dummies are in practice the same thing);
however, if you take stance (a), then your data is really (y-pmean(y)) in
hansl parlance, or if you prefer $y_{it} - \bar{y}_i$, and your model just
includes the betas, which are the coefficients of the deviations of the x
variables from their per-unit means. In this case, the relevant measure of
R^2 would be what plm reports and what stata calls the "within" R^2.
After discussing the issue for a bit, Allin and I thought it's probably
fair to report both, and simply drop the "adjusted" R^2, which makes
little sense in this context, considering that interpretations (a) and (b)
lead you to consider a different number of regressors, so you'd have to
define two parallel measures of \bar{R}^2 too. We'll commit the change in
CVS in a few hours.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
5:33 a.m.
Dear Riccardo,
Many thanks.
There are many fans of Gretl here at Ambedkar University (Delhi, India) and
your quick and thoughtful response has made our day.
Regards,
Jyotirmoy Bhattacharya
On Thu, Nov 28, 2013 at 12:22 AM, Riccardo (Jack) Lucchetti <
r.lucchetti(a)univpm.it> wrote:
On Wed, 27 Nov 2013, Jyotirmoy Bhattacharya wrote:
I was trying to understand how GRETL calculates R^2 for fixed-effects
> panel-data models as the value it produces is different from Stata and R's
> plm library.
>
Premise: I'm no big fan of R^2 in any shape or form, so consider my answer
as coming from someone who, in applied work, never even looks at the R^2
index.
However, it may be argued that a measure of the correlation between your
data and the prediction that the model gives you is a desirable descriptive
statistic to have. The problem here is: which model and which data.
The fixed-effect model could be conceived in two ways which are, in my
mind, equally defensible: (a) a nice, clean way to get rid of the
individual effects by using the fact that in the linear model a sufficient
stsistic is easy to compute or (b) as a clever way to estimate the
"important" parameters of a model in which you want to include (for some
reason) individual dummies. If you take perspective (b), then your data is
the unmodified y and your model includes the unit dummies as well, and
gretl is already doing the "right thing" (especially considering that it
could be useful, for instructional purposes, to show the students that FE
and a regression with lots of dummies are in practice the same thing);
however, if you take stance (a), then your data is really (y-pmean(y)) in
hansl parlance, or if you prefer $y_{it} - \bar{y}_i$, and your model just
includes the betas, which are the coefficients of the deviations of the x
variables from their per-unit means. In this case, the relevant measure of
R^2 would be what plm reports and what stata calls the "within" R^2.
After discussing the issue for a bit, Allin and I thought it's probably
fair to report both, and simply drop the "adjusted" R^2, which makes little
sense in this context, considering that interpretations (a) and (b) lead
you to consider a different number of regressors, so you'd have to define
two parallel measures of \bar{R}^2 too. We'll commit the change in CVS in a
few hours.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
9:33 a.m.
On Thu, 28 Nov 2013, Jyotirmoy Bhattacharya wrote:
Dear Riccardo,
Many thanks.
There are many fans of Gretl here at Ambedkar University (Delhi, India) and
your quick and thoughtful response has made our day.
Thanks for your interest and for spotting this. The reason why gretl gets
better day by day is because users are enormusly helpful for ironing out
all the shortcomings; so don't hesitate to do the same in the future
whenever necessary!
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3822
days inactive
3823
days old
3 comments
2 participants
participants (2)
-
Jyotirmoy Bhattacharya -
Riccardo (Jack) Lucchetti