10:29 a.m.
Hi,
this time I have a working example insofar as the following script
crashes gretl (recent cvs) for me:
<hansl>
open denmark
dataset addobs 4
# script operates on denmark with obs added until 1988:3
list lhs = IBO IDE LRM LRY
function void mycheck(list lhs, series rhs)
list allrhs = null
loop foreach i lhs
list rhs_$i = rhs(-1) # trivial example
if i=1
list allrhs = rhs_$i
else
list allrhs = allrhs ; rhs_$i
endif
endloop
system method=SUR
equations lhs allrhs
end system
fcast 1987:4 1988:3
end function
smpl 1974:1 1987:3
mycheck(lhs,LRM)
</hansl>
Maybe there is again a bug in the script, but I guess we agree that
gretl shouldn't crash.
Actually the background for testing this script was that I had the
following situation in a bigger, more realistic context:
1. do dynamic 3-step forecasts inside a function
2. do them sequentially in the sense of adding one obs at a time (for
forecast evaluation)
3. when the loop got to the penultimate available observation (in this
case sample end at 2010:10, so data available until 2010:11, but the
workfile range set until 2011:4 or so), the 3-step 'fcast' command only
produced forecasts for two periods, until 2010:12; the third row of the
$fcast output was simply empty!
4. furthermore, trying to access the third row of $fcast to copy the
value gave the error message "index 1 out of bounds" which is misleading
at best.
thanks and good luck hunting the bug,
sven
3:13 p.m.
On Mon, 12 Dec 2011, Sven Schreiber wrote:
Hi,
this time I have a working example insofar as the following script
crashes gretl (recent cvs) for me: [...]
Some groundwork that maybe the chief may find useful:
1) The following slightly simplified script exposes the problem too:
<hansl>
open denmark
dataset addobs 4
# script operates on denmark with obs added until 1988:3
list lhs = IBO IDE# LRM LRY
series rhs = LRM
list allrhs = null
loop foreach i lhs
list rhs_$i = rhs(-1) # trivial example
if i=1
list allrhs = rhs_$i
else
list allrhs = allrhs ; rhs_$i
endif
endloop
system method=SUR
equations lhs allrhs
end system
fcast 1987:4 1988:3
</hansl>
2) The problem seems to be tied to the fact that sys_add_fcast_variance()
is called at line 3777 in lib/src/system.c at a point when the
matrix sys->A is NULL. This, IMHO, causes subsequent problems in
sys_get_fcast_se(), sys_companion_matrix() etcetera. I guess sys->A should
have been initialised by then, but I don't have enough familiarity with
the system code to attempt a fix.
HTH,
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
6:49 p.m.
On Mon, 12 Dec 2011, Riccardo (Jack) Lucchetti wrote:
On Mon, 12 Dec 2011, Sven Schreiber wrote:
> Hi,
>
> this time I have a working example insofar as the following script
> crashes gretl (recent cvs) for me: [...]
Some groundwork that maybe the chief may find useful:
1) The following slightly simplified script exposes the problem too:
<hansl>
open denmark
dataset addobs 4
# script operates on denmark with obs added until 1988:3
list lhs = IBO IDE# LRM LRY
series rhs = LRM
list allrhs = null
loop foreach i lhs
list rhs_$i = rhs(-1) # trivial example
if i=1
list allrhs = rhs_$i
else
list allrhs = allrhs ; rhs_$i
endif
endloop
system method=SUR
equations lhs allrhs
end system
fcast 1987:4 1988:3
</hansl>
2) The problem seems to be tied to the fact that sys_add_fcast_variance() is
called at line 3777 in lib/src/system.c at a point when the matrix sys->A is
NULL.
Good detective work! Yes, the problem is that sys->A (the
matrix of cofficients on lagged left-hand side variables)
was NULL. There's now a fix in CVS so that we don't attempt
sys_add_fcast_variance() out-of-sample if this matrix is not
present.
Next question: why is sys->A not present? The trouble is that
gretl is not finding any lagged dependent variable in the
system, and the reason is that on the left we have series
named IBO, IDE, LRM, and LRY while on the right we have a
series named rhs_1, the first lag of a series named rhs. This
happens to have originated as a copy of LRM, but that fact is
not immediately apparent to gretl.
Here's a modified version of Sven's script that works with
current CVS. It requires passing the right-hand side variable
in a list, not as a plain series argument, so that gretl can
figure out its provenance.
<hansl>
function void mycheck(list lhs, list rhs)
list allrhs = null
loop foreach i lhs -q
list rhs_$i = lags(1, rhs) # trivial example
if i=1
list allrhs = const rhs_$i
else
list allrhs = allrhs ; const rhs_$i
endif
endloop
system method=SUR
equations lhs allrhs
end system
fcast 1987:4 1988:3
end function
open denmark
dataset addobs 4
list lhs = IBO IDE LRM LRY
list rhs = LRM
smpl 1974:1 1987:3
mycheck(lhs,rhs)
</hansl>
Allin
7:44 p.m.
On 12/12/2011 07:49 PM, Allin Cottrell wrote:
On Mon, 12 Dec 2011, Riccardo (Jack) Lucchetti wrote:
Good detective work! Yes, the problem is that sys->A (the
matrix of cofficients on lagged left-hand side variables)
was NULL. There's now a fix in CVS so that we don't attempt
sys_add_fcast_variance() out-of-sample if this matrix is not
present.
Thanks, I will test.
Next question: why is sys->A not present? The trouble is that
gretl is not finding any lagged dependent variable in the
system, and the reason is that on the left we have series
named IBO, IDE, LRM, and LRY while on the right we have a
series named rhs_1, the first lag of a series named rhs. This
happens to have originated as a copy of LRM, but that fact is
not immediately apparent to gretl.
Ok that explains why the forecast was performed until one period ahead
of the final available data.
However, what about the fact that happened to me that, quoting myself,
"trying to access the third row of $fcast to copy the value gave the
error message 'index 1 out of bounds'? Or does 'index 1' in fact mean
the first index, not that the index has the value one?
thanks,
sven
8:22 p.m.
On Mon, 12 Dec 2011, Sven Schreiber wrote:
On 12/12/2011 07:49 PM, Allin Cottrell wrote:
>
> Next question: why is sys->A not present? The trouble is that
> gretl is not finding any lagged dependent variable in the
> system, and the reason is that on the left we have series
> named IBO, IDE, LRM, and LRY while on the right we have a
> series named rhs_1, the first lag of a series named rhs. This
> happens to have originated as a copy of LRM, but that fact is
> not immediately apparent to gretl.
Ok that explains why the forecast was performed until one period ahead
of the final available data.
However, what about the fact that happened to me that, quoting myself,
"trying to access the third row of $fcast to copy the value gave the
error message 'index 1 out of bounds'? Or does 'index 1' in fact mean
the first index, not that the index has the value one?
Reading the post that you're quoting, I'm not getting a clear
enough sense of what you were doing to be able to replicate
it. I've tried some variants of accessing the $fcast matrix,
but while I can provoke an error by trying to read a row
that's not there, the message "index <whatever> out of bounds"
is always showing the row number that I requested (which is
what it's supposed to show).
Allin
10:23 p.m.
On 12/12/2011 09:22 PM, Allin Cottrell wrote:
On Mon, 12 Dec 2011, Sven Schreiber wrote:
> However, what about the fact that happened to me that, quoting
myself,
> "trying to access the third row of $fcast to copy the value gave the
> error message 'index 1 out of bounds'? Or does 'index 1' in fact
mean
> the first index, not that the index has the value one?
Reading the post that you're quoting, I'm not getting a clear
enough sense of what you were doing to be able to replicate
it. I've tried some variants of accessing the $fcast matrix,
but while I can provoke an error by trying to read a row
that's not there, the message "index <whatever> out of bounds"
is always showing the row number that I requested (which is
what it's supposed to show).
Well I'll keep trying to trigger it.
But in general, if gretl cannot figure out the lagged dependent
variables and thus cannot really do a forecast, shouldn't it throw an
error already at the 'fcast' command, and not "wait" until the user
tries to access the results via '$fcast'?
Thanks,
sven
1:29 a.m.
On Mon, 12 Dec 2011, Sven Schreiber wrote:
Well I'll keep trying to trigger it [a bad error message
on accessing a forecast-matrix row]
But in general, if gretl cannot figure out the lagged dependent
variables...
If you'll pardon me, gretl would have needed a crystal ball to
figure out that a lagged dependent variable was intended in the
script you posted: you passed the series in question by value.
and thus cannot really do a forecast, shouldn't it throw an error
already at the 'fcast' command, and not "wait" until the user
tries to access the results via '$fcast'?
In the case you posted, a one-step ahead forecast was possible but a
forecast over the full range specified was not. I'm not sure whether
it's preferable to flag an error, or just do what's possible. The
rows() function will tell you how many rows you got. But maybe we
should flag an error all the same.
Anyway, here's an attempt to reproduce something like what you
described (incremental forecasting); it works OK with CVS:
<hansl>
function matrix mfcast(list lhs, list rhs, int t2)
list allrhs = null
loop foreach i lhs -q
list rhs_$i = lags(1, rhs)
if i=1
allrhs = const rhs_$i
else
allrhs = allrhs ; const rhs_$i
endif
endloop
smpl ; t2
system method=SUR
equations lhs allrhs
end system --quiet
scalar fc1 = t2 + 1
scalar fc2 = t2 + 3
fcast fc1 fc2
matrix fc = $fcast
return fc
end function
open denmark
dataset addobs 3
list lhs = IBO IDE LRM LRY
list rhs = LRM
scalar t2 = 1987:1
loop 3 -q
matrix fc = mfcast(lhs,rhs,t2)
print fc
t2 += 1
endloop
</hansl>
Allin
10:20 a.m.
Am 13.12.2011 02:29, schrieb Allin Cottrell:
On Mon, 12 Dec 2011, Sven Schreiber wrote:
> Well I'll keep trying to trigger it [a bad error message
> on accessing a forecast-matrix row]
>
> But in general, if gretl cannot figure out the lagged dependent
> variables...
If you'll pardon me, gretl would have needed a crystal ball to
figure out that a lagged dependent variable was intended in the
script you posted: you passed the series in question by value.
Of course, and it wasn't meant as a criticism, just as "in those cases
where it is impossible for gretl to..."
> and thus cannot really do a forecast, shouldn't it throw an error
> already at the 'fcast' command, and not "wait" until the user
> tries to access the results via '$fcast'?
In the case you posted, a one-step ahead forecast was possible but a
forecast over the full range specified was not. I'm not sure whether
it's preferable to flag an error, or just do what's possible. The
rows() function will tell you how many rows you got. But maybe we
should flag an error all the same.
I'm sure there's a case for both ways. I would like to see at least a
warning though. And IMHO partial failure still is failure (again, not
gretl's fault, just a fact).
Anyway, here's an attempt to reproduce something like what you
described (incremental forecasting); it works OK with CVS:
Yes thanks, no need to do my work for me :-)
-sven
3:13 p.m.
New subject: "system"in functions (was Re: crash recipe)
On Tue, 13 Dec 2011, Sven Schreiber wrote:
Am 13.12.2011 02:29, schrieb Allin Cottrell:
> On Mon, 12 Dec 2011, Sven Schreiber wrote:
>
>> Well I'll keep trying to trigger it [a bad error message
>> on accessing a forecast-matrix row]
>>
>> But in general, if gretl cannot figure out the lagged dependent
>> variables...
>
> If you'll pardon me, gretl would have needed a crystal ball to
> figure out that a lagged dependent variable was intended in the
> script you posted: you passed the series in question by value.
Of course, and it wasn't meant as a criticism, just as "in those cases
where it is impossible for gretl to..."
OK, sorry if I seemed defensive.
> In the case you posted, a one-step ahead forecast was
> possible but a forecast over the full range specified was
> not. I'm not sure whether it's preferable to flag an error,
> or just do what's possible [...]
I'm sure there's a case for both ways. I would like to see at least a
warning though. And IMHO partial failure still is failure (again, not
gretl's fault, just a fact).
The thing is that with the "fcast" command in general, it may
well be that there are missing values to work around, so that
the output may also contain missing values -- which is not, in
general, considered an error in gretl. But I take your point
that a warning could be helpful.
> Anyway, here's an attempt to reproduce something like what
you
> described (incremental forecasting); it works OK with CVS
> [...]
Yes thanks, no need to do my work for me :-)
OK, but to revert to another issue you raised, I find that in
the body of a script the following trope works fine:
mysys <- system
equations Y X
end system
estimate mysys method=SUR
It doesn't work in a function, but that reflects the general
fact that the "name <- command" mechanism is not enabled in
functions. (Basically this is a GUI thing, meaning "create an
icon based on the command to the right of the "<-". And
functions can create icons only by returning bundles.)
But I'm thinking that while "<-" is not really appropriate in
functions we could support
system
equations Y X
end system
with the meaning: create an "anonymous" system which will then
be the target of the "estimate" (and "restrict") commands. The
system would stay in place as "the last defined system" until
either the function returns or another "system" command is
issued.
Allin
10:45 a.m.
New subject: "system"in functions (was Re: crash recipe)
On 12/13/2011 04:13 PM, Allin Cottrell wrote:
On Tue, 13 Dec 2011, Sven Schreiber wrote:
> Am 13.12.2011 02:29, schrieb Allin Cottrell:
>> Anyway, here's an attempt to reproduce something like
what you
>> described (incremental forecasting); it works OK with CVS
>> [...]
>
> Yes thanks, no need to do my work for me :-)
After updating to current cvs after the crash-bug fix, I have my
function working now, including the forecast. I don't remember changing
anything substantial that would make it easier for gretl to figure out
which are the lagged dependent variables though. (the small examples I
gave on the list shouldn't be overinterpreted here)
OK, but to revert to another issue you raised, I find that in
the body of a script the following trope works fine:
mysys <- system
equations Y X
end system
estimate mysys method=SUR
It doesn't work in a function, but that reflects the general
fact that the "name <- command" mechanism is not enabled in
functions. (Basically this is a GUI thing, meaning "create an
icon based on the command to the right of the "<-". And
functions can create icons only by returning bundles.)
Ah yes of course, I should have remembered that.
But I'm thinking that while "<-" is not really appropriate in
functions we could support
system
equations Y X
end system
with the meaning: create an "anonymous" system which will then
be the target of the "estimate" (and "restrict") commands. The
system would stay in place as "the last defined system" until
either the function returns or another "system" command is
issued.
This would be useful I think, especially since the estimation method
could possibly be passed to the function as an argument (string, or
integer options).
thanks,
sven
9:18 p.m.
New subject: "system"in functions (was Re: crash recipe)
On Wed, 14 Dec 2011, Sven Schreiber wrote:
On 12/13/2011 04:13 PM, Allin Cottrell wrote:
[on the question
of defining a "reusable" equation system
within a function]
>
> I'm thinking that while "<-" is not really appropriate in
> functions we could support
>
> system
> equations Y X
> end system
>
> with the meaning: create an "anonymous" system which will then
> be the target of the "estimate" (and "restrict") commands. The
> system would stay in place as "the last defined system" until
> either the function returns or another "system" command is
> issued.
This would be useful I think, especially since the estimation method
could possibly be passed to the function as an argument (string, or
integer options).
Here's a trivial example of what's now in CVS:
<hansl>
function void tester (list Y, list X)
system
equations Y X
end system
restrict $system
b[1,1] - b[2,1] = 1
end restrict
estimate $system method=sur
end function
open denmark
list lhs = IBO IDE
list rhs = const LRM LRY ; const LRM LRY
tester(lhs, rhs)
</hansl>
That is, if you give the "system" command in a function
without a "method" specification (in the first instance) the
system is saved internally, and can be accessed for the
purposes of "resrict" and "estimate" via "$system".
Allin
4798
days inactive
4800
days old
10 comments
3 participants
participants (3)
-
Allin Cottrell -
Riccardo (Jack) Lucchetti -
Sven Schreiber