Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Allin and Jack,
I’m a real trouble maker, it seems. :)
OK, here is a test file with the troubled data with only the hprice and inc variables. In my current gretl, it will read the data as string-valued series, and a freq on hprice will crash my gretl.
If I shorten the dataset, gretl will not read it and will give an error message instead, so I have to send you the full sample, which is much larger than the one I send before.
Fred
>
> Hi Jack,
> Attached is a small sample dataset with 3 string-valued variables, CITY, =
> hprice, & inc that crashes on CITY, hprice, but not on inc.
>
> Fred
>
>
>>
>> On Thu, 24 Dec 2020, Fred Engst wrote:
>>
>>> Hi Allin,
>>> Another repeatable crash today. This is from a dataset that my student =
>> gave me.
>>> After looking into the issue, I found that the problematic data series =
>> was actually read as a string-valued
>>> series that looked like numbers.
>>> Once I corrected this data issue, the frequency plot was fine.
>>> However, it still crashes on any string-valued data series.
>>> Attached is the crash report.
>>
>> This is weird. Normally, there is no problem with "freq" on string-valued=
>> =20
>> variables. Example:
>>
>> <hansl>
>> open grunfeld.gdt
>> freq ticker --plot=3Ddisplay
>> </hansl>
>>
>> There must be something funny in your student's dataset. Would you mind=20
>> sending it over? Thanks.
>
>
> --Apple-Mail=_F6A326C4-1187-46BA-BB14-9E5641F06218--
>
> ------------------------------
>
> Date: Thu, 24 Dec 2020 08:55:07 -0500 (EST)
> From: Allin Cottrell <cottrell(a)wfu.edu>
> Subject: [Gretl-users] Re: Repeatable crash on frequency distribution
> on a string-valued data series from GUI
> To: Gretl list <gretl-users(a)gretlml.univpm.it>
> Message-ID:
> <alpine.LFD.2.20.3.2012240850230.355041(a)myrtle.attlocal.net>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
> On Thu, 24 Dec 2020, Fred Engst wrote:
>
>> Another repeatable crash today. This is from a dataset that my
>> student gave me.
>>
>> After looking into the issue, I found that the problematic data
>> series was actually read as a string-valued series that looked
>> like numbers.
>>
>> Once I corrected this data issue, the frequency plot was fine.
>> However, it still crashes on any string-valued data series.
>> Attached is the crash report.
>
> Thanks, Fred. That's now fixed in git, snapshots to follow.
>
> Could you please send me the xlsx file that was imported to create
> the test3.gdt file that you posted? Besides the crashing problem
> itself there's something funny happening on importing those data: we
> end up with one string value consisting of 34 spaces, for hprice.
>
> Allin
>
> ------------------------------
>
> Date: Thu, 24 Dec 2020 16:20:18 +0100 (CET)
> From: "Riccardo (Jack) Lucchetti" <p002264(a)staff.univpm.it>
> Subject: [Gretl-users] Re: Repeatable crash on frequency distribution
> on a string-valued data series from GUI
> To: Gretl list <gretl-users(a)gretlml.univpm.it>
> Message-ID: <alpine.DEB.2.21.2012241618280.2199@newvaio>
> Content-Type: multipart/mixed;
> boundary="8323329-1436782324-1608823222=:2199"
>
> --8323329-1436782324-1608823222=:2199
> Content-Type: text/plain; charset=ISO-8859-15; format=flowed
> Content-Transfer-Encoding: quoted-printable
>
> On Thu, 24 Dec 2020, Allin Cottrell wrote:
>
>> On Thu, 24 Dec 2020, Fred Engst wrote:
>>
>>> Another repeatable crash today. This is from a dataset that my student=
> gave=20
>>> me.
>>> =20
>>> After looking into the issue, I found that the problematic data series=
> was=20
>>> actually read as a string-valued series that looked like numbers.
>>> =20
>>> Once I corrected this data issue, the frequency plot was fine.
>>> However, it still crashes on any string-valued data series.
>>> Attached is the crash report.
>>
>> Thanks, Fred. That's now fixed in git, snapshots to follow.
>>
>> Could you please send me the xlsx file that was imported to create the=20
>> test3.gdt file that you posted? Besides the crashing problem itself the=
> re's=20
>> something funny happening on importing those data: we end up with one s=
> tring=20
>> value consisting of 34 spaces, for hprice.
>
> I just pushed another commit to git to fix a bug exposed by the data file=
> =20
> that Fred sent. You'd get a segfault if you tried to apply the atof()=20
> function to the hprice series. Now this is fixed.
>
> Thanks, Fred!
>
> -------------------------------------------------------
> Riccardo (Jack) Lucchetti
> Dipartimento di Scienze Economiche e Sociali (DiSES)
>
> Universit=E0 Politecnica delle Marche
> (formerly known as Universit=E0 di Ancona)
>
> r.lucchetti(a)univpm.it
> http://www2.econ.univpm.it/servizi/hpp/lucchetti
> -------------------------------------------------------
> --8323329-1436782324-1608823222=:2199--
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Gretl-users mailing list -- gretl-users(a)gretlml.univpm.it
> To unsubscribe send an email to gretl-users-leave(a)gretlml.univpm.it
> Website: https://gretlml.univpm.it/postorius/lists/gretl-users.gretlml.univpm.it/
>
>
> ------------------------------
>
> End of Gretl-users Digest, Vol 167, Issue 30
> ********************************************
2 years, 11 months
Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Allin,
Another repeatable crash today. This is from a dataset that my student gave me.
After looking into the issue, I found that the problematic data series was actually read as a string-valued series that looked like numbers.
Once I corrected this data issue, the frequency plot was fine.
However, it still crashes on any string-valued data series.
Attached is the crash report.
Best,
Fred
2 years, 11 months
Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Jack,
Attached is a small sample dataset with 3 string-valued variables, CITY, hprice, & inc that crashes on CITY, hprice, but not on inc.
Fred
>
> On Thu, 24 Dec 2020, Fred Engst wrote:
>
>> Hi Allin,
>> Another repeatable crash today. This is from a dataset that my student =
> gave me.
>> After looking into the issue, I found that the problematic data series =
> was actually read as a string-valued
>> series that looked like numbers.
>> Once I corrected this data issue, the frequency plot was fine.
>> However, it still crashes on any string-valued data series.
>> Attached is the crash report.
>
> This is weird. Normally, there is no problem with "freq" on string-valued=
> =20
> variables. Example:
>
> <hansl>
> open grunfeld.gdt
> freq ticker --plot=3Ddisplay
> </hansl>
>
> There must be something funny in your student's dataset. Would you mind=20
> sending it over? Thanks.
2 years, 11 months
Book using Gretl
by 3J LEMA
Please suggest an econometric book that used Gretl?
Thank you.
2 years, 11 months
Updating variables
by Brian Revell
Is there any way to update variables that are functions of another, if the
original variable Y has been amended or updated . The former is
straightforward to edit Y's values. But I am not clear that for example,
l-Y is automatically recalculated. And if l-Y has already been used in a
session, Gretl will not let it be deleted giving the error message cannot
be deleted variable in use..
Suggestions around this issue welcomed
Brian J Revell
Professor Emeritus
Agricultural and Food Economics
Harper Adams University
Newport
Shropshire TF10 8NB
Tel 01952 815237
Tel: +44 1952 728153
Mbl +44 7976 538712
University: +44 1952 815235
alt: email: bjrevell(a)harper-adams.ac.uk
2 years, 11 months
State space model with missing observations
by Filipe Costa
Hi community!
I'm trying to compute a time-varying state space model to calculate stocks' betas. Unfortunately, I have some missing observations and, when this is the case, the model implemented by Gretl suddenly stops and doesn't compute anything (as expected). My question is simple: Is there a way of instructing Gretl to ignore missing values and compute the state spacel model?
Here is the example code taken from chapter 35 of Gretl manual. Please note that I added line 6 "INFQ[20] = NA" in order to have a missing data point in the data.
<hansl>
function void at_each_step(bundle *b)
b.obsymat = transp(b.mX[b.t,])
end function
open AWM.gdt --quiet
smpl 1974:1 1994:1
INFQ[20] = NA # added this line to have a missing value
/* parameter initialization */
scalar b0 = mean(INFQ)
scalar s_obs = 0.1
scalar s_state = 0.1
/* bundle setup */
bundle B = ksetup(INFQ, 1, 1, 1)
matrix B.mX = {URX}
matrix B.depvar = {INFQ}
B.timevar_call = "at_each_step"
B.diffuse = 1
/* ML estimation of intercept and the two variances */
mle LL = err ? NA : B.llt
B.obsy = B.depvar - b0
B.obsvar = s_obs^2
B.statevar = s_state^2
err = kfilter(&B)
params b0 s_obs s_state
end mle
/* display the smoothed time-varying slope */
ksmooth(&B)
series tvar_b1hat = B.state[,1]
series tvar_b1se = sqrt(B.stvar[,1])
gnuplot tvar_b1hat --time-series --with-lines --output=display \
--band=tvar_b1hat,tvar_b1se,1.96 --band-style=fill
</hansl>
Best,
Filipe
2 years, 11 months
OLS/WLS comparison
by Artur Bala
Hi,
Not sure if this is a gretl issue or an econometric one actually.
I'm trying to come up with a script that replicates Stata's "rreg" command
which is basically a robust estimation for outliers correction. It is based
on an iterative WLS resulting in an optimal weight vector.
Now, for testing purposes, I'm running OLS and WLS models successively. I
note that the WLS' "Statistics based on the original data" SSR and S.E of
regression are slightly different from the respective OLS' output (*in bold
*below). Should these parts of outputs be the same?
If not, hhat is the difference coming from ?
Best,
Artur
? ols crime X
Model 1: OLS, using observations 1-50
Dependent variable: crime
...
Mean dependent var 566.6600 S.D. dependent var 295.8773
*Sum squared resid 2442333 S.E. of regression 227.9573*
...
? wls W crime X
Model 2: WLS, using observations 1-50
Dependent variable: crime
Variable used as weight: W
...
Statistics based on the weighted data:
Sum squared resid 1925024 S.E. of regression 202.3807
...
Statistics based on the original data:
Mean dependent var 566.6600 S.D. dependent var 295.8773
*Sum squared resid 2469623 S.E. of regression 229.2273*
2 years, 11 months
Reading large csv and sorting data set -- a comparison with 2 python libs
by Artur Tarassow
Hi all,
out of curiosity I tried to replicate a comparison between two python
libraries, namely pandas and py-polars. I also added gretl to the horse
race.
There two simple tasks to do:
1) Load a 360MB csv file with two columns and 26 million rows.
2) Sort by one of the columns.
The summary is as follows:
1) Py-polars takes about 2.5 seconds to load a 360MB large csv file as a
data frame. Pandas is about 5 times slower while Gretl needs 36 seconds
for this.
2) Sorting 26 million records takes py-polars about 4.6 seconds and
hence is just slightly faster than Pandas with 5.8 sec. -- ok, still
20% percent. Gretl needs about 14 seconds for the same task.
I am not too bothered with gretl's performance -- okay, reading a csv
may be faster but still: how often do you load such a big file?
The repo and readme can be accessed here:
https://github.com/atecon/gretl_pandas_pypolars
There is one thing which is weird though: at the end of the README.md
you see that I tried to store the loaded csv data set as a gdtb file.
But after an hour of computation I gave up... Maybe worth to look at.
Best,
Artur
2 years, 11 months
Var autoregression
by Brian Revell
Hi
A warning.
the Model Var Autoregression in the GUI can crash out Gretl -no error
message re estimation no. of variables too few or anything -it just
disappears from the screen. So all preceding work done without the outputs
saved in the session (not necessarily only from previous Var
autoregressions ) will be lost. Mine is the latest Gretl version.
Brian J Revell
Professor Emeritus
2 years, 11 months
