12:54 p.m.
Hi guys,
I love gretl, but I have a problem.
When I run the builtin fixed effect model on a panel data with over 5000 firms, the gretl
does fine.
But since the dependent variable has over 90% zeros, I tried to do a Tobit model with
entity dummies (i.e. fixed effect tobit). Once I included over 5000 dummies using the
gretl's GUI, gretl crashed with an error from the crash report: "Application
Specific Information: detected buffer overflow”.
I then ran a fixed effect tobit by gretlcli from the command line and it gets stuck for 3
days without much progress.
Just now I tried to run a fixed effect by including entity dummies on OLS through GUI, it
crashed gretl once again, with the same crash report.
Next, I tried fixed effect by including du_* on the OLS command through gretlcli, it was
able to complete in a few minutes, much longer than the builtin panel model with the
—fixed-effects option.
I’m a bit puzzled, however. If a panel fixed effect works fine, why can’t it work for a
tobit model? Or it just will take a very long time to complete due to the nonlinearity?
Is there a way to do the fixed effect like the builtin panel way rather than the explicit
including dummies way on a tobit model?
In either case, the buffer overflow should be looked at.
Thanks in advance for all your help.
Fred
12:59 p.m.
Hi Fred,
In either case, the buffer overflow should be looked at.
Of course, gretl should not crash. Could you specify which gretl version
on which OS you're using please? If you aren't using the latest stable
version gretl 2019a, could you please try this one again.
I can't say anything more on the Logit issue -- sorry.
Best,
Artur
1:20 p.m.
Am 09.02.2019 um 13:54 schrieb Fred Engst:
Hi guys,
I love gretl, but I have a problem.
When I run the builtin fixed effect model on a panel data with over 5000 firms, the gretl
does fine.
But since the dependent variable has over 90% zeros, I tried to do a Tobit model with
entity dummies (i.e. fixed effect tobit).
Hi, the econometrics of this is problematic. Quoting from Stata's
documentation for xttobit: "There is no command for a parametric
conditional fixed-effects model, as there does not exist a sufficient
statistic allowing the fixed effects to be conditioned out of the
likelihood." and: "Unconditional fixed-effects tobit models may be fit
with the tobit command with indicator variables for the panels; [...]
However, unconditional fixed-effects estimates are biased."
So you're trying to perform a biased estimation. Sometimes there may be
good reasons to do this, but I guess what you really want is
random-effects Tobit.
Unfortunately, AFAIK there is no native support nor a function package
that does that yet. It is also not a trivial estimation algorithm in
terms of numerics.
Just now I tried to run a fixed effect by including entity dummies on
OLS through GUI, it crashed gretl once again, with the same crash report.
Please clarify, do you mean you did a plain OLS estimation in panel data
and adding a lot of unit dummies (literal fixed effects), and then it
crashed? This would of course be a bug.
Next, I tried fixed effect by including du_* on the OLS command
through gretlcli, it was able to complete in a few minutes, much longer than the builtin
panel model with the —fixed-effects option.
This is not unexpected, because the standard way of almost all
implementations is to demean the variables first (along the T-dimension,
the within transformation). Instead adding the many unit dummies is
theoretically possible but has numerical problems.
Is there a way to do the fixed effect like the builtin panel way
rather than the explicit including dummies way on a tobit model?
If you're prepared to do a biased estimation you might apply the within
transformation to the variables manually and then estimate a standard
Tobit model on the transformed variables. However, it's not clear to me
how to preserve the zeros in the dependent variable then...
In either case, the buffer overflow should be looked at.
Absolutely!
thanks for the report,
Sven
2:11 p.m.
On Sat, 9 Feb 2019, Sven Schreiber wrote:
Am 09.02.2019 um 13:54 schrieb Fred Engst:
> Hi guys,
> I love gretl, but I have a problem.
> When I run the builtin fixed effect model on a panel data with over 5000
> firms, the gretl does fine.
> But since the dependent variable has over 90% zeros, I tried to do a Tobit
> model with entity dummies (i.e. fixed effect tobit).
Hi, the econometrics of this is problematic. Quoting from Stata's
documentation for xttobit: "There is no command for a parametric conditional
fixed-effects model, as there does not exist a sufficient statistic allowing
the fixed effects to be conditioned out of the likelihood." and:
"Unconditional fixed-effects tobit models may be fit with the tobit command
with indicator variables for the panels; [...] However, unconditional
fixed-effects estimates are biased."
Yes, this is the well-known "incidental parameters problem". Of course you
can switch to a random-effects specification, but then you should be
prepared to defend several extra assumptions. Without having given this
much thought, I believe this shouldn't be difficult to do via GHK.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:23 p.m.
On Sat, 9 Feb 2019, Riccardo (Jack) Lucchetti wrote:
Without having given this much thought, I believe this shouldn't
be
difficult to do via GHK.
Actually, Guass-Hermite integration is the proper tool for this job. See
the quadtable() function.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
12:11 p.m.
On Sat, 9 Feb 2019, Riccardo (Jack) Lucchetti wrote:
On Sat, 9 Feb 2019, Riccardo (Jack) Lucchetti wrote:
> Without having given this much thought, I believe this shouldn't be
> difficult to do via GHK.
Actually, Guass-Hermite integration is the proper tool for this job. See the
quadtable() function.
Ok, so I put together a small script that performs ML of random-effects
tobit via Gauss-Hermite integration (attached). It would be nice to find
some kind soul that turns this into a function package.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:21 a.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
Am 11.02.2019 um 13:11 schrieb Riccardo (Jack) Lucchetti:
On Sat, 9 Feb 2019, Riccardo (Jack) Lucchetti wrote:
> On Sat, 9 Feb 2019, Riccardo (Jack) Lucchetti wrote:
>
>
>> Without having given this much thought, I believe this shouldn't be
>> difficult to do via GHK.
>
> Actually, Guass-Hermite integration is the proper tool for this job.
> See the quadtable() function.
Ok, so I put together a small script that performs ML of
random-effects tobit via Gauss-Hermite integration (attached). It
would be nice to find some kind soul that turns this into a function
package.
Microeconometricians to the front!
Thanks Jack, your script runs fine here. Actually I'm a bit surprised
that the functions and the script aren't very long. Good for gretl (and
for us)!
Perhaps it would be good to put this into the feature request tracker
and attach the code there. So far the tracker has mainly been used for
requests concerning core gretl, but I'd say that's no iron law.
thanks,
sven
11:06 a.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
On Tue, 12 Feb 2019, Sven Schreiber wrote:
Microeconometricians to the front!
Thanks Jack, your script runs fine here. Actually I'm a bit surprised that
the functions and the script aren't very long. Good for gretl (and for us)!
A "proper" function package would involve a bit more coding. For example,
panel statistics, proper prinout of the results, etcetera. Also,
analytical derivatives wouldn't hurt (although that would be a bit more
complicated, I think). And then, of course documentation.
But even a simple wrap-up of what's already there would be better than
nothing, I guess.
Perhaps it would be good to put this into the feature request tracker
and
attach the code there. So far the tracker has mainly been used for requests
concerning core gretl, but I'd say that's no iron law.
Agreed.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:47 a.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
Am 12.02.2019 um 12:06 schrieb Riccardo (Jack) Lucchetti:
On Tue, 12 Feb 2019, Sven Schreiber wrote:
>
> Microeconometricians to the front!
But even a simple wrap-up of what's already there would be better than
nothing, I guess.
> Perhaps it would be good to put this into the feature request tracker
> and attach the code there. So far the tracker has mainly been used for
> requests concerning core gretl, but I'd say that's no iron law.
Agreed.
Agreed in the sense that you're going to do it or that it's no iron law?
-s
8:11 a.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
On Thu, 14 Feb 2019, Sven Schreiber wrote:
Am 12.02.2019 um 12:06 schrieb Riccardo (Jack) Lucchetti:
> On Tue, 12 Feb 2019, Sven Schreiber wrote:
>
>>
>> Microeconometricians to the front!
>
> But even a simple wrap-up of what's already there would be better than
> nothing, I guess.
>
>> Perhaps it would be good to put this into the feature request tracker and
>> attach the code there. So far the tracker has mainly been used for
>> requests concerning core gretl, but I'd say that's no iron law.
>
> Agreed.
Agreed in the sense that you're going to do it or that it's no iron law?
Iron law. :)
As for the request tracker, I don't know how to do it. If you could see to
it, that'd be great. Otherwise, it'll try to get to it as soon as I can.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
1:20 p.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
Am 14.02.2019 um 09:11 schrieb Riccardo (Jack) Lucchetti:
On Thu, 14 Feb 2019, Sven Schreiber wrote:
> Am 12.02.2019 um 12:06 schrieb Riccardo (Jack) Lucchetti:
>> On Tue, 12 Feb 2019, Sven Schreiber wrote:
>>> Perhaps it would be good to put this into the feature
request
>>> tracker and attach the code there. So far the tracker has mainly
>>> been used for requests concerning core gretl, but I'd say that's no
>>> iron law.
>>
>> Agreed.
>
> Agreed in the sense that you're going to do it or that it's no iron law?
Iron law. :)
As for the request tracker, I don't know how to do it. If you could see
to it, that'd be great. Otherwise, it'll try to get to it as soon as I can.
Well, it's a web-based clickable interface and pretty much
self-explanatory. Compared to finding stuff in the git C source it is
very easy!
Anyway, this is now ticket 117:
https://sourceforge.net/p/gretl/feature-requests/117/
cheers,
sven
1:40 p.m.
New subject: Random-effects Tobit (Re: Fixed Effect crashed ...)
On Thu, 14 Feb 2019, Sven Schreiber wrote:
Well, it's a web-based clickable interface and pretty much
self-explanatory.
Ah ok, thanks for that!
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
2152
days inactive
2157
days old
11 comments
4 participants
participants (4)
-
Artur T. -
Fred Engst -
Riccardo (Jack) Lucchetti -
Sven Schreiber