12:29 p.m.
I'd like to put to people an issue that Jack and I have been
discussing lately, and that seems to require a
backward-incompatible change at some point.
By way of background, recall that user-defined functions in gretl
are supposed to respect the principle of encapsulation.
Basically, this means that
* external variables are accessible within a function only insofar
as they are passed as arguments; and
* no variables are changed at the caller level other than via
assignment of a return value offered by a function.
In addition, settings of program parameters using the "set"
command, within a function, are confined to that function: we save
the values at the caller level and restore them on exit from the
function. And when a model is estimated within a function, this
does not displace the "last model" (the target for accessors such
as $coeff) from the point of view of the caller: this will be the
last model estimated before the function was called.
So far so good, but the trouble is that the current handling of
lists, both as arguments to functions and as return values, is not
wholly consistent with encapsulation. Sven noted one aspect of
this a while back; Jack noticed another aspect recently.
(1) Sven's point: Suppose a function constructs and returns a list
of variables. As a toy example, suppose it returns a list holding
the cubes of the variables given in a list argument. As a
function writer, one might construct the names of the variables in
the return list as <varname_i>_3, where <varname_i> represents the
name of the variable at position i in the input list.
The issue is this: it's true that nothing will be changed at the
caller level unless this return value is assigned, but all the
same such assignment may have an unintended effect. If there
already exists, say, a variable named "x_3", and if a variable "x"
is passed to the cube function, then x_3 will be overwritten by
the cube of x.
I would argue that this is not an insurmountable problem. I think
we can avoid excessive trouble by ensuring that any public gretl
function packages that do this sort of thing are very explicit in
their documentation. We could add a required field to gretl
function packages that return lists -- a warning along the lines
of "This function returns a list of variables named on the pattern
<pattern>. Please note that any existing variables of the same
name will be overwritten."
(2) Jack's point: Consider a function that accepts a list as an
argument, and suppose that the function is called, with a list
argument that includes a series named "x".
Now let's say the writer of this function wants to create a local
scalar called "x". What happens? You'll get a type error, from
trying to overwrite a series with a scalar. There can be only one
variable named "x" inside the function, and the series x is made
"visible" within the function by virtue of its inclusion in a list
argument.
And what happens if the function writer tries to create a local
series named "x"? If the list argument is marked "const" you'll
get an error; if not, the external "x" will be silently
over-written. Bad.
A moment's thought shows that this problem is less tractable than
the first: it's a design flaw, and it can't be solved simply by
documenting clearly what the function does.
Here's a proposal for solving the second problem. It is
implemented in CVS but not by default, only if you edit
lib/src/gretl_func.c and change the definition
#define PROTECT_LISTS 0
to
#define PROTECT_LISTS 1
The essential component is that, when a series is passed to a
function as part of a list argument, the series is _not_
visible/accessible by name within the function. In relation to
the problem cases mentioned above, it will then be OK for the
function writer to create a local scalar or series "x": this won't
collide with a list-argument series "x" since the latter will
never by "seen" by name within the function.
Accordingly, we modify the action of a "foreach i" loop over the
members of a list argument to a function. Instead of "$i"
retrieving the name of the series at position i in the list, it
retrieves the ID number of that series (and series ID numbers are
always unique, regardless of where we are in terms of function
execution).
Two questions are likely to arise here:
1. How do you construct a new series name based on the name of a
series given via a list argument (as in the cubes example)?
Answer: You can get the input variable's name as a string variable
using the built-in varname() function, with the variable's ID
number as argument.
2. How do you get hold of the values of a series given via a list
argument, for use on the right-hand side of a genr expression?
Answer: Use the new built-in function varcopy(). Again, this
takes a variable's ID number as argument, circumventing the
possible collision of names.
Illustration: Take the trivial cubes example -- here is a
new-style implementation
function make_cubes (list xlist)
list retlist = null
loop foreach i xlist
string oldname = varname($i)
sprintf newname "%14s_3", oldname
series @newname = varcopy($i)^3
setinfo @newname -d "cube of @oldname"
list retlist += @newname
end loop
return list retlist
end function
Two questions for you: First, does this solution seem acceptable?
Second, if it's OK, how soon should we aim to make the change?
Allin.