Re: Negation operator
by Gordon Hughes
You are correct. I can reproduce the problem but I had not
identified the source accurately.
Unfortunately I had transferred the script file from Windows to Linux
for testing with the latest CVS version. In doing so, there appears
to have been a problem with an end-of-line character which meant that
the following "endif" statement was getting lost or
concatenated. That happened for the version that I was testing with
"if a != b" but not for the reversed version with "if a = b".
This should be a warning for anyone who transfers script files
between Windows & Linux.
Gordon
>Are you sure? The following script runs fine here:
>
><script>
>nulldata 10
>a = 1
>b = 2
>if a != b
> printf "NOT equal!\n"
>else
> printf "Equal.\n"
>endif
></script>
>
>Riccardo (Jack) Lucchetti
17 years, 1 month
Negation operator
by Gordon Hughes
Forgive me if this has already been spotted, but I think that
something has gone wrong with the negation operator in the latest
CVS. I have recompiled gretl this morning [1.7.6rc1, 28/7/08] and
found that I had repeated problems with the syntax if !isnull(var).
After beating my head against a brick wall I have established that
if a = b
..
else
..
endif
works fine, but
if a != b
..
else
..
endif
generates an error message reporting that there is an "unmatched if".
Gordon
17 years, 1 month
Re: optional arguments
by Gordon Hughes
I am sorry that I have been rather slow in following up the earlier
discussion, but I have been preoccupied with another project.
Yes, I had worked out the mechanism that you describe for providing
optional arguments. However, it does not work for all argument
types. Also, the order of the arguments is important. To explain:
A. The current version (1.7.6rc1) won't accept the following syntax
in an argument list
function xxx( ... string *var[null], ... )
which means that string arguments cannot be omitted. As far as I can
work out, it is valid to use this syntax for integers, scalars,
matrices and series, but not for lists and strings. Given the recent
discussion of list arguments, the first is probably sensible but it
seems arbitrary to rule out the possibility of null string arguments.
B. The argument list is read from left to right - fair enough but it
means that if a user provides 3 arguments for a function with, say, 5
potential arguments, then these 3 arguments are mapped to argument 1,
argument 2, argument 3. This is entirely reasonable for the
programmer but less obvious for the novice user who wants to define
arguments 1, 2 & 5. Of course, what they must do is specify function
xxx (arg1, arg2, null, null, arg3) but much of the simplicity has
been lost unless the programmer can guess how the function is most
likely to be used. That is really my point about the convenience of
using $accessors rather than pointer arguments.
I have also thought some more about my functions. On reflection, it
seems to be sensible to retain separate functions for certain
purposes - e.g. the calculation of residuals because a user may wish
to do this for observations that were not part of the estimation
sample. On the other hand, I would also like to allow a user to
supply initial parameter values because some models have serious
problems with convergence. Hence, what one really needs is a
flexible front-end function that can be used to control calls to one
or more helper functions that have more rigid requirements but which
the average user doesn't need to know about. At the moment that is
difficult, but it may be worth bearing in mind as a long term goal.
Finally, with respect to access to functions through the GUI my
reaction is that the key thing is to flag the defaults clearly - ie
what happens if arguments are omitted. This should be done on the
main page, so that users would know what will happen if they do not
define any arguments on the 2nd optional page of the
notebook. However, I am not the best judge of this because I rely
almost entirely upon script or command files in order to have a
record of what I have done.
Gordon
>There's something that's present in the manual but maybe not
>obvious. If you're writing a function that provides some "basic"
>output but also offers optional info that's likely to be of
>interest only to experts, the trick is to (a) make the extra info
>available via pointer arguments, (b) give those pointers a default
>value of "null", and (c) make sure they come at the end of the
>list of parameters. Then the user does not have to mess with them
>at all.
>
>For example, a function with this signature
>
> function myfun (list L, matrix M,
> matrix *A[null], matrix *B[null])
>
>might simply be called as, say,
>
> myfun(xlist, Xmat)
>
>since trailing arguments that have default values can be omitted
>altogether. So, if I'm understanding your point right, you really
>shouldn't need two versions of the function.
>
>In the gretl GUI, the call dialog for such a function (assuming it
>has been packaged) will show the pointer arguments, with the
>"null" default in place. One possible refinement of the GUI would
>be to separate optional pointer arguments in such a way that
>things don't look "too complicated" to the average user. For
>instance, the function call dialog could have a "notebook"
>structure, with such arguments placed under a second tab.
>
>If that sounds useful, I'll have a go at implementing it.
>
>Allin.
17 years, 1 month
RFC: functions and lists in gretl 1.7.6
by Allin Cottrell
I'm at the point of revising the User's Guide on the topic of
list arguments to functions, and this task gives a different
perspective on what we've been discussing.
Brief recap: We've discovered a nasty problem with the way we've
handled list arguments to user-defined functions, up till now.
We have a fix for this but it involves a backward-incompatible
change. In CVS we have a boolean switch, "protect_lists", which
flips between the old behavior and the fixed behavior. The
question is: what should be the default value for this switch?
Right now, the old behavior is the default.
Now, consider presenting this in the documentation.
1) "We have discovered a serious problem with the handling of list
arguments to functions. This is now fixed as of gretl 1.7.6.
Unfortunately the fix involves a backward-incompatible change.
For a limited period, however, you can revert to the old behavior
by doing..."
2) "We have discovered a serious problem with the handling of list
arguments to functions. We have a fix for this -- sort of, but
it's not activated by default since it involves a
backward-incompatible change. You can 'preview' the fix by
doing..."
Which of these sounds better?
In addition, how do we present the rules governing the use of list
arguments within functions? If "protect_lists" is off by default,
it seems to me the primary exposition has to be of the old, broken
rules, with a note added to describe the new ones.
But if "protect_lists" is on, we can give pride of place to the
new, correct rules, and add a note on how old functions should be
modified to conform to them.
I feel that these considerations argue quite strongly for making
the change now (i.e. making "protect_lists" on by default in
gretl 1.7.6).
Your comments, please!
Allin.
17 years, 1 month
Re: gretl functions and lists
by Gordon Hughes
At 00:33 15/07/2008, you wrote:
>I agree the extension you propose would be nice. However, the underlying
>machinery which deals with the dollar accessors doesn't lend itself to
>accommodate it easily. Maybe using pointers is easier. Here's a small
>proof-of-concept script:
>
><script>
>function reverse_ols(series y, list X, matrix *rcoeff, series *rres)
> ols y X --quiet
> rcoeff = -$coeff
> rres = -$uhat
>end function
>
>open data4-1
>list reg = const sqft bedrms baths
>ols price reg
>u = $uhat
>matrix foo
>series bar
>reverse_ols(price, reg, &foo, &bar)
>print u bar -o
></script>
>
>Riccardo (Jack) Lucchetti
>Dipartimento di Economia
>Università Politecnica delle Marche
>
>r.lucchetti(a)univpm.it
>http://www.econ.univpm.it/lucchetti
The difficulty with the use of pointers rather
than $ accessors is really one of how to provide
a single package which implements the basic
procedure for the novice user while giving the
expert user the ability to recover additional
results. By adding the requirement to define
additional variables and provide the relevant
pointers the function becomes less easy to use
for someone who just wants to run a simple
estimation and has no requirement for anything
beyond the basic printed output. Still, it may
be the case that my goal is not achievable with
the way in which functions and function packages work.
As an alternative, I am inclined to provide two
versions of each function package - one for
novice users with no results being returned, and
the other for those who want to get access to
coefficients, etc. Both would be little more
than different front-ends to the same basic
procedures but with alternative sets of options
and information returned. It might be helpful if
there were some default conventions which would
allow a number of function packages to follow
broadly the same approach and naming protocols.
At the same time, it would be nice to be able to
use the $ accessors for compatibility with normal gretl commands.
Gordon Hughes
17 years, 1 month
more user-function news
by Allin Cottrell
In current CVS and Windows snapshot:
* We now have the boolean "set" variable "protect_lists", for use
with list arguments to functions. It's off by default.
* "const-ness" is now handled correctly for arguments taking the
form of pointers to scalars, series and matrices. This property
is "inherited", so you'll (correctly) get an error from this sort
of thing:
<script>
function baz (matrix *M)
M = mnormal(2,2) # bzzt!
end function
function foo (const matrix *M)
baz(&M)
end function
matrix M = I(3)
foo(&M)
</script>
Although function baz does not mark its matrix-pointer argument as
const, the const guarantee given by function foo carries over.
Allin.
17 years, 1 month
Standard error of OLS residuals
by mamarini＠istat.it
Hi,
estimating a simple OLS I get a different standard error than expected.
The formula is simply ((res'*res)/n)^(1/2), isn't it? Making the calculation
outside Gretl I achieve a different result.
The version is 1.7.5 released last June.
Thanks!
Marco Marini
17 years, 1 month
PROTECT_LIST: first casualties (long)
by Riccardo (Jack) Lucchetti
Consider the following script:
<script>
set echo off
set messages off
function foo(list X)
printf "\nInside function foo:\n"
loop foreach i X -q
printf "$i\n"
end loop
end function
function bar(list X)
printf "\nInside function bar:\n"
loop foreach i X -q
printf "%s\n", varname(i)
end loop
end function
nulldata 10
x1 = normal()
x2 = normal()
list X = const x1 x2
printf "\nOutside a function:\n"
loop foreach i X -q
printf "$i\n"
end loop
foo(X)
bar(X)
</script>
With current CVS, with PROTECT_LISTS=1, it yields:
<output>
Outside a function:
const
x1
x2
Inside function foo:
const
2
3
Inside function bar:
index
x1
x2
</output>
There are several points I'd like to raise: bear with me, this is going to
be neither short nor painless.
Looking at the output of "foo", clearly the constant is treated
specially, which should not happen. This also affects the output of "bar",
in which varname(i) becomes varname(const)=varname(1)="index"
Even if the above particular asymmetry is removed, another one becomes
evident: foreach loops inside functions behave differently from those
outside. In the latter case, "$i" yields a string; in the former, an
integer. This has all sort of weird consequences: for example, a statement
like "genr $i_xxx = normal()" is legal outside a function but becomes
illegal inside, since identifiers cannot start with numbers.
A possible way out would be to endow variables passed through lists with
local aliases, like for example "X->const", or "X.x1", in which we use
some "illegal" marker ("->" or "." in the above examples) to make sure
that no name clashes inside functions can occur.
This isn't trivial to implement, and has the added disadvantage that you
may end up with monsters like "LongNameList.VeryVerboseVariable", but the
alternatives are not very palatable either: with the new mechanism (which,
I repeat, I consider indispensable), either we're prepared to live with a
syntax inconsistency (terrible) or we change the meaning of $i in foreach
loops outside functions. This would be a huge backward-incompatible change
(are you there, Sven?). In other words, if the "local name" solution (plan
A) proves impractical, I propose to extend the varname() mechanism for
retrieving variable names to all foreach loops (plan B). The number of
existing scripts that will be broken if plan B is carried forward is
doubtlessly enormous, but I think it would be a sad necessity.
Now, the question is: what should our policy be?
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
17 years, 1 month
Re: gretl functions and lists
by Gordon Hughes
I would like to respond to Allin's note by drawing upon my experience
of writing a variety of functions over the last 2-3 weeks. In doing
this I was aware of the dangers of overwriting existing variables
when returning a list of variables created in the functions. Hence,
I would certainly favour the separation implied by the adoption of
the PROTECT_LISTS 1 option.
However, I find the explanation and/or the handling of lists passed
as function arguments rather clumsy. My point concerns bullet 2
close to the end of the note plus the code fragment for
make_cubes. It appears to imply that whenever one passes a list of
series containing, say, the independent variables for an estimation
function it will be necessary to include a loop over the list to copy
the contents of this list to new variables inside the function. Yet,
the principle of encapsulation should mean that the list "xlist" in
make_cubes should be a copy of the list provided as the argument, so
the varcopy ought to be redundant in these circumstances. Further,
writing the loop at the beginning of any function would be a
discouragement for novice users, among whom I would count myself
though providing the necessary code in the documentation would reduce this.
But, I have a further point that I would like to raise. When I was
writing my stochastic frontier functions I found that it was
necessary to write two different public functions to carry out what
is essentially one task, which is tedious and incurs unnecessary
overheads of function calls, etc. The reason is that I wanted to
return two types of information: (a) estimation results including
parameter coefficients, variance-covariance matrix and
log-likelihood, etc, and (b) residuals or the like (actually the
inefficiency estimates). Item (a) can be fitted into a matrix whose
dimension depends upon the number of independent variables, while
item (b) is a series or list of series whose length is equal to the
number of observations in the sample. So, the first function returns
a matrix of estimation results, while the second function returns a
list of residuals, etc. Because the different sets of information
have different dimensions, it is not practical to return it in a
single matrix or list.
In the case of gretl's own estimation procedures this information is
contained in $coeff, etc. Would it be possible to provide a series
of function placeholders such as $func_coeff, $func_uhat or
$func_ret1-$func_ret9 which the writer of the function can fill, if
desired, and the program calling the function can access after the
function has been called. This would deal with Sven's problem
because one could prohibit a function from returning a list - a
matrix or a series is not a problem because the matrix or series
returned by the function is copied to the new matrix or series in the
calling program. Instead, the list created by my SFA efficiency
function can be replaced by a single execution of the SFA estimation
function which would put the residuals and efficiency estimates in
$func_ret1, ... for retrieval by the user if wanted. Of course, this
information would be overwritten by subsequent function calls, but
this is already the case for $coeff, etc.
Gordon
17 years, 1 month
PROTECT_LISTS on
by Riccardo (Jack) Lucchetti
For anyone who's interested: since I consider the "inner naming" problem
the most serious we've had in gretl since I can remember (short summary in
Allin's message of a few days ago:
http://lists.wfu.edu/pipermail/gretl-devel/2008-July/001234.html), I've
been running gretl with PROTECT_LIST=1 for a few days and had no problems
of any sort. I plan to give it a good test run between tomorrow and next
Monday/Tuesday. If no problem arises, I would say it needs to be in for
1.7.6.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
17 years, 1 month
