[Gretl-users] Fixed Effect crashed when I tried to include over 5000 entity dummies
Hi Fred,
> In either case, the buffer overflow should be looked at.
Of course, gretl should not crash. Could you specify which gretl version
on which OS you're using please? If you aren't using the latest stable
version gretl 2019a, could you please try this one again.
I can't say anything more on the Logit issue -- sorry.
Best,
Artur
Hi Artur,
here is the top part of my crash report:
------------------------------------------------------------------------------------
Process: gretl [10753]
Path: /Applications/Gretl.app/Contents/Resources/bin/gretl
Identifier: net.sourceforge.gretl
Version: 2019a-git
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: gretl [10753]
User ID: 502
Date/Time: 2019-01-17 16:54:27.499 +0800
OS Version: Mac OS X 10.11.6 (15G22010)
Report Version: 11
Anonymous UUID: 76CACE37-6884-3AB3-F91C-F52618C62272
Sleep/Wake UUID: 0AD2F5AA-ECE2-4687-894D-0FE3111E22A0
Time Awake Since Boot: 160000 seconds
Time Since Wake: 30000 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY
Application Specific Information:
detected buffer overflow
-------------------------------------------------------------------------------------
I don’t suppose you need the rest of the report.
Fred
> Just now I tried to run a fixed effect by including entity dummies on OLS through
GUI, it crashed gretl once again, with the same crash report.
Please clarify, do you mean you did a plain OLS estimation in panel data
and adding a lot of unit dummies (literal fixed effects), and then it
crashed? This would of course be a bug.
> Next, I tried fixed effect by including du_* on the OLS command through gretlcli, it
was able to complete in a few minutes, much longer than the builtin panel model with the
?fixed-effects option.
This is not unexpected, because the standard way of almost all
implementations is to demean the variables first (along the T-dimension,
the within transformation). Instead adding the many unit dummies is
theoretically possible but has numerical problems.
> Is there a way to do the fixed effect like the builtin panel way rather than the
explicit including dummies way on a tobit model?
If you're prepared to do a biased estimation you might apply the within
transformation to the variables manually and then estimate a standard
Tobit model on the transformed variables. However, it's not clear to me
how to preserve the zeros in the dependent variable then...
> In either case, the buffer overflow should be looked at.
Absolutely!
thanks for the report,
Sven
Hi Sven,
1. Yes, you are right. I did a plain OLS estimation in panel data and adding a lot of unit
dummies and then it crashed.
The crash report is identical to the one above other than the crash date.
2. I didn’t realize that is how gretl does the fixed effect, for it reported LSDV
R-squared, which I thought means it included the contribution of those dummies.
3. I really don’t know much or thought much about “fixed effect” tobit. I just thought by
adding those dummies I don’t have to demean those dependent variables. Otherwise they will
lose their non-nagetive nature, making tobit a misfit. Now I realized that to estimate
over 5000 parameters is next to impossible anyway.
It seems that if I reduce the detentions of those “fixed effects” to a few
characteristics, say less than 10, then the tobit model should work.
Thanks so much for your reply.
Fred