print command
by Artur Bala
Hi,
Two questions related to the "print" command.
1) Is the "range" option supposed to work jointly with the "by-obs" option?
When by-obs is omitted the full range of series is printed out despite the
specification for range. For example:
? dataset sortby absr1std
? print absr1std --byobs --range=46:50
absr1std
ME 1.426741
MT 1.460884
VT 1.742409
DC 2.616447
FL 2.902663
? print absr1std --range=46:50
Full data range: 1 - 51 (n = 51)
0.0805403 0.0908925 0.0932311 0.0987927 0.127675 0.143004 0.164553
0.193521 0.203738 0.257765 0.280333 0.304542 0.307398 0.352839
... # I'm cutting here!!
? dataset sortby state
2) I'd like to print out the 5 greatest values of a series. As you may see
above, I first sort the whole dataset, then use print with range option and
finally sort again the dataset to revert to the initial position. Ok, these
operations come with no effort at all but, and this is just a suggestion,
would it be more intuitive if a sortby option be included in the print
command? Or a "dummy" option?
Best,
Artur
4 years
Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Allin and Jack,
I’m a real trouble maker, it seems. :)
OK, here is a test file with the troubled data with only the hprice and inc variables. In my current gretl, it will read the data as string-valued series, and a freq on hprice will crash my gretl.
If I shorten the dataset, gretl will not read it and will give an error message instead, so I have to send you the full sample, which is much larger than the one I send before.
Fred
>
> Hi Jack,
> Attached is a small sample dataset with 3 string-valued variables, CITY, =
> hprice, & inc that crashes on CITY, hprice, but not on inc.
>
> Fred
>
>
>>
>> On Thu, 24 Dec 2020, Fred Engst wrote:
>>
>>> Hi Allin,
>>> Another repeatable crash today. This is from a dataset that my student =
>> gave me.
>>> After looking into the issue, I found that the problematic data series =
>> was actually read as a string-valued
>>> series that looked like numbers.
>>> Once I corrected this data issue, the frequency plot was fine.
>>> However, it still crashes on any string-valued data series.
>>> Attached is the crash report.
>>
>> This is weird. Normally, there is no problem with "freq" on string-valued=
>> =20
>> variables. Example:
>>
>> <hansl>
>> open grunfeld.gdt
>> freq ticker --plot=3Ddisplay
>> </hansl>
>>
>> There must be something funny in your student's dataset. Would you mind=20
>> sending it over? Thanks.
>
>
> --Apple-Mail=_F6A326C4-1187-46BA-BB14-9E5641F06218--
>
> ------------------------------
>
> Date: Thu, 24 Dec 2020 08:55:07 -0500 (EST)
> From: Allin Cottrell <cottrell(a)wfu.edu>
> Subject: [Gretl-users] Re: Repeatable crash on frequency distribution
> on a string-valued data series from GUI
> To: Gretl list <gretl-users(a)gretlml.univpm.it>
> Message-ID:
> <alpine.LFD.2.20.3.2012240850230.355041(a)myrtle.attlocal.net>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
> On Thu, 24 Dec 2020, Fred Engst wrote:
>
>> Another repeatable crash today. This is from a dataset that my
>> student gave me.
>>
>> After looking into the issue, I found that the problematic data
>> series was actually read as a string-valued series that looked
>> like numbers.
>>
>> Once I corrected this data issue, the frequency plot was fine.
>> However, it still crashes on any string-valued data series.
>> Attached is the crash report.
>
> Thanks, Fred. That's now fixed in git, snapshots to follow.
>
> Could you please send me the xlsx file that was imported to create
> the test3.gdt file that you posted? Besides the crashing problem
> itself there's something funny happening on importing those data: we
> end up with one string value consisting of 34 spaces, for hprice.
>
> Allin
>
> ------------------------------
>
> Date: Thu, 24 Dec 2020 16:20:18 +0100 (CET)
> From: "Riccardo (Jack) Lucchetti" <p002264(a)staff.univpm.it>
> Subject: [Gretl-users] Re: Repeatable crash on frequency distribution
> on a string-valued data series from GUI
> To: Gretl list <gretl-users(a)gretlml.univpm.it>
> Message-ID: <alpine.DEB.2.21.2012241618280.2199@newvaio>
> Content-Type: multipart/mixed;
> boundary="8323329-1436782324-1608823222=:2199"
>
> --8323329-1436782324-1608823222=:2199
> Content-Type: text/plain; charset=ISO-8859-15; format=flowed
> Content-Transfer-Encoding: quoted-printable
>
> On Thu, 24 Dec 2020, Allin Cottrell wrote:
>
>> On Thu, 24 Dec 2020, Fred Engst wrote:
>>
>>> Another repeatable crash today. This is from a dataset that my student=
> gave=20
>>> me.
>>> =20
>>> After looking into the issue, I found that the problematic data series=
> was=20
>>> actually read as a string-valued series that looked like numbers.
>>> =20
>>> Once I corrected this data issue, the frequency plot was fine.
>>> However, it still crashes on any string-valued data series.
>>> Attached is the crash report.
>>
>> Thanks, Fred. That's now fixed in git, snapshots to follow.
>>
>> Could you please send me the xlsx file that was imported to create the=20
>> test3.gdt file that you posted? Besides the crashing problem itself the=
> re's=20
>> something funny happening on importing those data: we end up with one s=
> tring=20
>> value consisting of 34 spaces, for hprice.
>
> I just pushed another commit to git to fix a bug exposed by the data file=
> =20
> that Fred sent. You'd get a segfault if you tried to apply the atof()=20
> function to the hprice series. Now this is fixed.
>
> Thanks, Fred!
>
> -------------------------------------------------------
> Riccardo (Jack) Lucchetti
> Dipartimento di Scienze Economiche e Sociali (DiSES)
>
> Universit=E0 Politecnica delle Marche
> (formerly known as Universit=E0 di Ancona)
>
> r.lucchetti(a)univpm.it
> http://www2.econ.univpm.it/servizi/hpp/lucchetti
> -------------------------------------------------------
> --8323329-1436782324-1608823222=:2199--
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Gretl-users mailing list -- gretl-users(a)gretlml.univpm.it
> To unsubscribe send an email to gretl-users-leave(a)gretlml.univpm.it
> Website: https://gretlml.univpm.it/postorius/lists/gretl-users.gretlml.univpm.it/
>
>
> ------------------------------
>
> End of Gretl-users Digest, Vol 167, Issue 30
> ********************************************
4 years
Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Allin,
Another repeatable crash today. This is from a dataset that my student gave me.
After looking into the issue, I found that the problematic data series was actually read as a string-valued series that looked like numbers.
Once I corrected this data issue, the frequency plot was fine.
However, it still crashes on any string-valued data series.
Attached is the crash report.
Best,
Fred
4 years
Repeatable crash on frequency distribution on a string-valued data series from GUI
by Fred Engst
Hi Jack,
Attached is a small sample dataset with 3 string-valued variables, CITY, hprice, & inc that crashes on CITY, hprice, but not on inc.
Fred
>
> On Thu, 24 Dec 2020, Fred Engst wrote:
>
>> Hi Allin,
>> Another repeatable crash today. This is from a dataset that my student =
> gave me.
>> After looking into the issue, I found that the problematic data series =
> was actually read as a string-valued
>> series that looked like numbers.
>> Once I corrected this data issue, the frequency plot was fine.
>> However, it still crashes on any string-valued data series.
>> Attached is the crash report.
>
> This is weird. Normally, there is no problem with "freq" on string-valued=
> =20
> variables. Example:
>
> <hansl>
> open grunfeld.gdt
> freq ticker --plot=3Ddisplay
> </hansl>
>
> There must be something funny in your student's dataset. Would you mind=20
> sending it over? Thanks.
4 years
Book using Gretl
by 3J LEMA
Please suggest an econometric book that used Gretl?
Thank you.
4 years
Updating variables
by Brian Revell
Is there any way to update variables that are functions of another, if the
original variable Y has been amended or updated . The former is
straightforward to edit Y's values. But I am not clear that for example,
l-Y is automatically recalculated. And if l-Y has already been used in a
session, Gretl will not let it be deleted giving the error message cannot
be deleted variable in use..
Suggestions around this issue welcomed
Brian J Revell
Professor Emeritus
Agricultural and Food Economics
Harper Adams University
Newport
Shropshire TF10 8NB
Tel 01952 815237
Tel: +44 1952 728153
Mbl +44 7976 538712
University: +44 1952 815235
alt: email: bjrevell(a)harper-adams.ac.uk
4 years
State space model with missing observations
by Filipe Costa
Hi community!
I'm trying to compute a time-varying state space model to calculate stocks' betas. Unfortunately, I have some missing observations and, when this is the case, the model implemented by Gretl suddenly stops and doesn't compute anything (as expected). My question is simple: Is there a way of instructing Gretl to ignore missing values and compute the state spacel model?
Here is the example code taken from chapter 35 of Gretl manual. Please note that I added line 6 "INFQ[20] = NA" in order to have a missing data point in the data.
<hansl>
function void at_each_step(bundle *b)
b.obsymat = transp(b.mX[b.t,])
end function
open AWM.gdt --quiet
smpl 1974:1 1994:1
INFQ[20] = NA # added this line to have a missing value
/* parameter initialization */
scalar b0 = mean(INFQ)
scalar s_obs = 0.1
scalar s_state = 0.1
/* bundle setup */
bundle B = ksetup(INFQ, 1, 1, 1)
matrix B.mX = {URX}
matrix B.depvar = {INFQ}
B.timevar_call = "at_each_step"
B.diffuse = 1
/* ML estimation of intercept and the two variances */
mle LL = err ? NA : B.llt
B.obsy = B.depvar - b0
B.obsvar = s_obs^2
B.statevar = s_state^2
err = kfilter(&B)
params b0 s_obs s_state
end mle
/* display the smoothed time-varying slope */
ksmooth(&B)
series tvar_b1hat = B.state[,1]
series tvar_b1se = sqrt(B.stvar[,1])
gnuplot tvar_b1hat --time-series --with-lines --output=display \
--band=tvar_b1hat,tvar_b1se,1.96 --band-style=fill
</hansl>
Best,
Filipe
4 years
OLS/WLS comparison
by Artur Bala
Hi,
Not sure if this is a gretl issue or an econometric one actually.
I'm trying to come up with a script that replicates Stata's "rreg" command
which is basically a robust estimation for outliers correction. It is based
on an iterative WLS resulting in an optimal weight vector.
Now, for testing purposes, I'm running OLS and WLS models successively. I
note that the WLS' "Statistics based on the original data" SSR and S.E of
regression are slightly different from the respective OLS' output (*in bold
*below). Should these parts of outputs be the same?
If not, hhat is the difference coming from ?
Best,
Artur
? ols crime X
Model 1: OLS, using observations 1-50
Dependent variable: crime
...
Mean dependent var 566.6600 S.D. dependent var 295.8773
*Sum squared resid 2442333 S.E. of regression 227.9573*
...
? wls W crime X
Model 2: WLS, using observations 1-50
Dependent variable: crime
Variable used as weight: W
...
Statistics based on the weighted data:
Sum squared resid 1925024 S.E. of regression 202.3807
...
Statistics based on the original data:
Mean dependent var 566.6600 S.D. dependent var 295.8773
*Sum squared resid 2469623 S.E. of regression 229.2273*
4 years
Reading large csv and sorting data set -- a comparison with 2 python libs
by Artur Tarassow
Hi all,
out of curiosity I tried to replicate a comparison between two python
libraries, namely pandas and py-polars. I also added gretl to the horse
race.
There two simple tasks to do:
1) Load a 360MB csv file with two columns and 26 million rows.
2) Sort by one of the columns.
The summary is as follows:
1) Py-polars takes about 2.5 seconds to load a 360MB large csv file as a
data frame. Pandas is about 5 times slower while Gretl needs 36 seconds
for this.
2) Sorting 26 million records takes py-polars about 4.6 seconds and
hence is just slightly faster than Pandas with 5.8 sec. -- ok, still
20% percent. Gretl needs about 14 seconds for the same task.
I am not too bothered with gretl's performance -- okay, reading a csv
may be faster but still: how often do you load such a big file?
The repo and readme can be accessed here:
https://github.com/atecon/gretl_pandas_pypolars
There is one thing which is weird though: at the end of the README.md
you see that I tried to store the loaded csv data set as a gdtb file.
But after an hour of computation I gave up... Maybe worth to look at.
Best,
Artur
4 years
