6:37 p.m.
Hi all,
sorry, this is going to be long and a bit involved. While working on a
package, I encountered some weirdness when creating new gretl list
objects off of a $model coming from a GUI model window after doing OLS.
I do have a stripped-down package test.gfn which I'm attaching for
illustration.
* Technical aim here: Inside the GUI function (here: GUI_uniFCboot)
which is run from the Analysis menu of the OLS model window, I want to
look at the regressors $model.xlist and I want to focus only on those
terms that are exogenous (not lagged endogenous, and also not the constant).
* Example:
1) Install somewhere the test.gfn package, attach it to the menu, load
the denmark dataset, and in the GUI regress LRM on IBO, const, and two
own lags.
2) Go to the new menu entry Analysis/testtesttest, choose horizon=2 (but
that shouldn't matter) and hit OK.
3) I get my debug printout: "We shouldn't have any lags remaining in
here: IBO,LRM_1,LRM_2", which obviously *does* contain the LRM lags.
* Background: In the function, I'm trying to create a sub-list by doing
(where 'mod' is a copy of $model):
list Lexo = mod.xlist - lags(mod.maxlag, mod.ylist) - const # all but
endo-lags and const
and then I'm printing out varname(Lexo), with the result above. Now I
can theoretically imagine some reasons why my approach doesn't work,
related to the list ID numbers inside bundles and functions and so on.
* HOWEVER, the real weirdness comes when I add/uncomment the following
lines right before the Lexo creation, namely:
bundle br = rhslags($model) # especially this line seems to make a
difference # although it shouldn't!
print br.ylagnames
(The package version with these lines active is enclosed as test2.gfn.)
-- And then everything works, i.e., there aren't any lagged endogenous
terms left in the subsequent Lexo creation; result: "We shouldn't have
any lags remaining in here: IBO" !
The point here is that the call to the private function rhslags() should
be completely redundant and without any effect, shouldn't it? The return
bundle br isn't used here, and there are no pointers involved. Instead
it magically makes it so that everything falls into place, at least in
this example.
What's behind all this? Some gretl bug? Or just further mystery with
bundled series and lists? And concretely, how can I successfully remove
from $model.xlist the lag terms in this context without doing black magic?
This is with gretl 2025a on Windows 11.
thanks a lot for reading until the end
sven
2:53 p.m.
On Sun, 6 Apr 2025, Sven Schreiber wrote:
Hi all,
sorry, this is going to be long and a bit involved. While working on a
package, I encountered some weirdness when creating new gretl list objects
off of a $model coming from a GUI model window after doing OLS. I do have a
stripped-down package test.gfn which I'm attaching for
illustration.[...]
Thanks for the report.
So Sven has in mind a function package which offers attachment to
the "Analysis" menu in a gretl model window. It's intended to
perform an action for which a particular list of series is needed,
namely, the list of regressors for the model in question, minus any
lags of the dependent variable (and also minus the constant).
A function in this context has access to the $model accessor and
Sven wishes to construct the needed list thus:
bundle mod = $model
list Lexo = mod.xlist - lags(mod.maxlag, mod.ylist) - const
where the call "lags(mod.maxlag, mod.ylist)" is meant to create the
lagged dependent list to subtract from mod.xlist, which contains all
the regressors.
Now here's the dataset context in the example Sven describes.
* The "denmark" dataset contains the four series LRM, LRY, IBO and
IDE, with IDs 1 to 4.
* A regression of the form "ols LRM const IBO LRM(-1 to -2)" is run,
which results in the addition to the dataset of the lags LRM_1 and
LRM_2, with series IDs 5 and 6.
So mod.xlist holds "1 0 3 5 6" and the lagged dependent list is the
subset "5 6".
Now what happens when the packaged function is called? It turns out
that the call "lags(mod.maxlag, mod.ylist)" results in the
(re-)creation of the two LRM lags, appended to the dataset as series
7 and 8. The original versions, under IDs 5 and 6, are not "visible"
within the function because they were not passed as arguments (nor
via a standard list, but just inside a bundle). So the list
subtraction doesn't work: gretl tries to drop series 7 and 8 from
mod.xlist, but they're not present.
It seems that it ought to be possible to ensure that when a
list-containing bundle is passed as a function argument, the listed
series become "visible" -- then Sven's approach would work. However,
in the meantime here's an approach that does work. We define a
helper function as follows
<hansl>
function list exog_list (list xlist, list ylist)
string yl = varnames(ylist)[1] ~ "_"
strings xnames = varnames(xlist)
list drop
loop i=1..nelem(xlist)
if strstr(xnames[i], yl) != ""
drop += xlist[i]
endif
endloop
list Lexo = xlist - drop - const
return Lexo
end function
</hansl>
and in the primary function we do
bundle mod = $model
list Lexo = exog_list(mod.xlist, mod.ylist)
Allin
6:21 p.m.
On Sun, Apr 6, 2025 at 2:37 PM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
Hi all,
sorry, this is going to be long and a bit involved. While working on a
package, I encountered some weirdness when creating new gretl list
objects off of a $model coming from a GUI model window after doing OLS.
Here's a further response to Sven after doing some more investigation.
It's unlikely to be of interest to anyone who's not in the business of
writing packaged functions that key off models represented in windows
in the gretl GUI.
The key question is this: under what conditions does gretl, inside a
function, "know about" series that exist at the level from which the
function was called? The answer:
1) Obviously, gretl knows about series that were explicitly passed as arguments.
2) In addition, gretl knows about series that are referenced in a
"list" passed as an argument. Such series are temporarily "localized"
to the function, then returned to their original level when the
function returns.
3) As an extension of point 2, we also temporarily "localize" series
that are referenced in a list contained in a bundle passed as an
argument to the function in question.
And that's all. What Sven was trying to do would work if and only if
we also "localize" series referenced in a list contained in a bundle
that's _not_ passed as an argument but is made available within the
function via the $model accessor -- something we have not attempted to
date.
Allin
6:25 p.m.
Am 07.04.2025 um 16:53 schrieb Allin Cottrell:
Now here's the dataset context in the example Sven describes.
* The "denmark" dataset contains the four series LRM, LRY, IBO and
IDE, with IDs 1 to 4.
* A regression of the form "ols LRM const IBO LRM(-1 to -2)" is run,
which results in the addition to the dataset of the lags LRM_1 and
LRM_2, with series IDs 5 and 6.
So mod.xlist holds "1 0 3 5 6" and the lagged dependent list is the
subset "5 6".
Now what happens when the packaged function is called? It turns out
that the call "lags(mod.maxlag, mod.ylist)" results in the
(re-)creation of the two LRM lags, appended to the dataset as series 7
and 8. The original versions, under IDs 5 and 6, are not "visible"
within the function because they were not passed as arguments (nor via
a standard list, but just inside a bundle). So the list subtraction
doesn't work: gretl tries to drop series 7 and 8 from mod.xlist, but
they're not present.
Thanks, Allin. Yes, I also did some digging into those lists and saw
this phenomenon.
I'm not sure I fully understand that "visibility" is the actual problem.
Note the example in the packaging guide for the (otherwise unrelated)
list-maker facility. The similar line there is "list L = m.xlist -
m.instlist", so all those bundled lists and their members seem to be
visible alright? It feels more like gretl doesn't recognize the lag
property of the former series.
Anyway, the packaging guide reminded me that not only the $model
accessor is available in the model window context, but the $xlist
accessor should be, too. Maybe using that would be more successful? I
will test...
It seems that it ought to be possible to ensure that when a
list-containing bundle is passed as a function argument, the listed
series become "visible" -- then Sven's approach would work. However,
in the meantime here's an approach that does work. We define a helper
function as follows
<hansl>
function list exog_list (list xlist, list ylist)
string yl = varnames(ylist)[1] ~ "_"
strings xnames = varnames(xlist)
list drop
loop i=1..nelem(xlist)
if strstr(xnames[i], yl) != ""
drop += xlist[i]
endif
endloop
list Lexo = xlist - drop - const
return Lexo
end function
</hansl>
Thanks for this concrete workaround, Allin. I think I have used the
series names for comparisons like this in other contexts in the past as
well, which is a bit like a brute-force attack.
Here's a version of your function without an explicit loop, which I hope
is equivalent:
<hansl>
function list exog_list (list xlist, list ylist)
string yl = varnames(ylist)[1] ~ "_"
strings xnames = varnames(xlist)
strings xnamesini = substr(xnames, 1, strlen(yl)) # like "LRM_"
matrix where = instrings(xnamesini, yl)
list drop = xlist[where]
list Lexo = xlist - drop - const
return Lexo
end function
</hansl>
Not sure what the minimum gretl requirement is for these constructs, though.
thanks
sven
6:33 p.m.
Am 07.04.2025 um 20:21 schrieb Cottrell, Allin:
Here's a further response to Sven after doing some more
investigation.
It's unlikely to be of interest to anyone who's not in the business of
writing packaged functions that key off models represented in windows
in the gretl GUI.
The key question is this: under what conditions does gretl, inside a
function, "know about" series that exist at the level from which the
function was called? The answer:
1) Obviously, gretl knows about series that were explicitly passed as arguments.
2) In addition, gretl knows about series that are referenced in a
"list" passed as an argument. Such series are temporarily
"localized"
to the function, then returned to their original level when the
function returns.
3) As an extension of point 2, we also temporarily "localize" series
that are referenced in a list contained in a bundle passed as an
argument to the function in question.
And that's all.
Hm, what about $xlist, $uhat and all that in a packaged function from a
model window, not accessed through $model but directly in their own
right? Would that count as (3b)?
What Sven was trying to do would work if and only if
we also "localize" series referenced in a list contained in a bundle
that's _not_ passed as an argument but is made available within the
function via the $model accessor -- something we have not attempted to
date.
OK, but two questions remaining:
1) See my other email some minutes ago: the listmaker example with list
"L = m.xlist - m.instlist" seems to do just that, no?
2) There's still the weirdness of my OP that it did work when I called
my other function rhslags() which shouldn't have changed anything AFAICS.
-s
8:35 p.m.
On Mon, 7 Apr 2025, Sven Schreiber wrote:
Am 07.04.2025 um 20:21 schrieb Cottrell, Allin:
> Here's a further response to Sven after doing some more investigation.
> It's unlikely to be of interest to anyone who's not in the business of
> writing packaged functions that key off models represented in windows
> in the gretl GUI.
>
> The key question is this: under what conditions does gretl, inside a
> function, "know about" series that exist at the level from which the
> function was called? The answer:
>
> 1) Obviously, gretl knows about series that were explicitly passed as
> arguments.
>
> 2) In addition, gretl knows about series that are referenced in a
> "list" passed as an argument. Such series are temporarily
"localized"
> to the function, then returned to their original level when the
> function returns.
>
> 3) As an extension of point 2, we also temporarily "localize" series
> that are referenced in a list contained in a bundle passed as an
> argument to the function in question.
>
> And that's all.
Hm, what about $xlist, $uhat and all that in a packaged function from a model
window, not accessed through $model but directly in their own right? Would
that count as (3b)?
No, I'm afraid not. Membership in a list that is not provided as an
argument either directly or via a bundle argument, does not confer
visibility on "external" series inside a function.
> What Sven was trying to do would work if and only if
> we also "localize" series referenced in a list contained in a bundle
> that's _not_ passed as an argument but is made available within the
> function via the $model accessor -- something we have not attempted to
> date.
OK, but two questions remaining:
1) See my other email some minutes ago: the listmaker example with list "L =
m.xlist - m.instlist" seems to do just that, no?
No, that's just a matter of subtracting one given list (vector of
ints) from another. It doesn't require any access to the actual
series content. Your example requires such access, if gretl is to
realize that series 5 and 6 already contain the requested lags of
LRM.
2) There's still the weirdness of my OP that it did work when I
called my
other function rhslags() which shouldn't have changed anything AFAICS.
By passing the $model bundle as an argument to another (arbitrary)
function you made its list contents visible, and they remained
visible after that function returned.
Allin
12:45 p.m.
Am 07.04.2025 um 22:35 schrieb Allin Cottrell:
On Mon, 7 Apr 2025, Sven Schreiber wrote:
>
> OK, but two questions remaining:
>
> 1) See my other email some minutes ago: the listmaker example with
> list "L = m.xlist - m.instlist" seems to do just that, no?
No, that's just a matter of subtracting one given list (vector of
ints) from another. It doesn't require any access to the actual series
content. Your example requires such access, if gretl is to realize
that series 5 and 6 already contain the requested lags of LRM.
OK, that's
understandable.
> 2) There's still the weirdness of my OP that it did work when I
> called my other function rhslags() which shouldn't have changed
> anything AFAICS.
By passing the $model bundle as an argument to another (arbitrary)
function you made its list contents visible, and they remained visible
after that function returned.
Now that really is black magic! Indeed, I just tested this by creating
the following totally useless function
function void fdummy(const bundle mod)
print "dummy here"
end function
and then in my GUI-based function I issued the call 'fdummy($model)' --
as you said, everything works nicely after that.
I'm always happy to learn, but this strikes me as really weird. Also,
you wrote "(arbitrary) function", so I replaced the call to my fdummy
function with a call to a built-in, as in 'getkeys($model)'. Then it was
back to square one, failing again! So is it just arbitrary
_user-defined_ functions?
thanks
sven
14
days inactive
18
days old
6 comments
3 participants
participants (3)
-
Allin Cottrell -
Cottrell, Allin
-
Sven Schreiber