minor issues
by Sven Schreiber
Hello,
with 1.7.6rc2 I experience the following minor flaws. The timing of this report is purely coincidental, I'm not a big fan of last-minute fixes without proper testing, so I'm not pushing this to be fixed before 1.7.6.
1) Clicking ok in the dialog window for generating random variables adds the specified variable, but doesn't close the window. Actually I can imagine that this is a feature, not a bug, but just in case it's not...
2) With panel --random-effects estimation the Hausman test of RE vs. FE isn't printed when the test statistic becomes negative. From what I recall of the translated strings, there used to be a warning message instead. I didn't agree with the content of that warning message, but whatever one thinks of those negative statistics, gretl shouldn't just silently ignore them and leave the user wondering why no test output is shown.
3) And finally a related feature request: I think $hausman should not only access the endogeneity tests after tsls, but should also retrieve the Hausman test after --random-effects.
thanks,
sven
17 years
Re: Missing values in mle and functions - setobs issue
by Gordon Hughes
I can supplement my previous query with a report based on my
experiments, which may be an error or a feature. A skeleton example
is as follows:
function xpanel(x)
genr dtest1=$datatype
print dtest1
# Should print 3.000 for panel data
genr xunit=$unit
genr time
setobs 1 1 --cross-section
genr dtest2=$datatype
print dtest2
# Should print 1.000 as data is now cross-section
smpl ok(x) --restrict
...
return series y
setobs xunit time --panel-vars
genr dtest3=$datatype
print dtest3
# Should print 3.000 for panel data
end function
Now call xpanel using a dataset that has been defined as a panel dataset:
genr test1=$datatype
print test1
# Should print 3.000 for panel data
series y = xpanel(x)
genr test2=$datatype
print test2
# Should print 3.000 for panel data but actually the result is 1.000
What is happening is that it is possible to redefine the datatype
from panel to cross-section within the function and operate on the
(sampled) cross-section data. Further, the setobs command at the end
appears to allow you to go back to the original panel
structure. However, that does not get stick when you goes back to
the original calling routine and you are left with cross-section data
at the higher level .
Gordon
17 years
Re: Missing values in mle and functions
by Gordon Hughes
Thank you - this solves part of the problem. However, I am still
left with the consequence of the way in which gretl handles panel
data. I am trying to ensure that my panel functions can handle
unbalanced panels, but this seems to be very difficult.
When gretl reads an unbalanced panel it automatically creates a
balanced panel by creating new cases with missing data. After that,
smpl refuses to omit cases that would produce an unbalanced
panel. But, this may be perfectly acceptable in some circumstances -
eg suppose I want to estimate a stochastic frontier using data for
Company A for 1995-2004, Company B for 1992-2006, Company C for 1990-2001, etc.
At the moment, the only option seems to be: within the function (a)
create my own internal panel time and unit variables, (b) tell gretl
to treat the data as a cross-section, not a panel, (c) now drop
missing data and estimate the model without using any of gretl's
panel facilities.
So far, so good but I have found that the command "setobs 1 1
--cross-section" within a function is not purely local to that
function and its dependents. It changes the way that the dataset is
handled on exit from the function and I am not clear what will happen
in all circumstances if I try to restore the panel data structure at
the end of the function. Is there any other way round this problem?
Gordon
> > A. Is this the right thing to do? Deleting cases with missing values in
> > calculating the log-likelihood, etc is not difficult, but there may be
> > models when missing values cause problems if not dealt with properly - eg
> > any lag structure. Would it be possible to provide an option (--delmiss)
> > for case-wise deletion of missing values?
>
>In many cases, a very handy technique is using the ok() function with a
>list argument, as in
><script>
>list X = y x1 x2
>smpl ok(X) --restrict
></script>
>whose meaning should be obvious.
>
>You also have another way to achieve the same result, which is arguably
>more byzantine but perhaps more transparent in cases when what you really
>need is a linear combination of some variables for the valid observations
>only:
>
><script>
>nulldata 20
>x = (uniform()>0.8) ? NA : normal()
>list X = const x
>b = {1;1}
>y = (uniform()>0.8) ? NA : normal()
>e = y - lincomb(X, b)
>print y x e -o
>smpl ok(e) --restrict
>print y x e -o
></script>
17 years
mwrite strangeness
by Sven Schreiber
well sorry, here comes another:
? mwrite(results800er,"results800er.mat")
produces a file w/o extension ("results800er"), but
? mwrite(results800er,"results800er.huhu")
preserves the entire string, as I intended. At first I thought it was because windows might not show known file extensions, but I always have that option off and I just checked it's still off. So somehow gretl seems to eat the ".mat" part.
cheers,
sven
17 years
Missing values in mle and functions
by Gordon Hughes
I would like to pose some questions about the best way of dealing
with missing values when using mle. I had assumed, wrongly, that mle
simply applies case-wise deletion to cases with missing
values. Instead it terminates with a more or less informative error
message. On the other hand, built-in estimation commands seem to
handle missing values internally.
A. Is this the right thing to do? Deleting cases with missing
values in calculating the log-likelihood, etc is not difficult, but
there may be models when missing values cause problems if not dealt
with properly - eg any lag structure. Would it be possible to
provide an option (--delmiss) for case-wise deletion of missing values?
B. In writing a function it is not difficult to test for missing
values, but then what should the function do? Here the question is
what are the consequences of adjusting the sample to exclude cases
with missing values within the function? Would this affect the
sample used in the program that calls the function? If so, one would
really need a "smpl --restore" command to set the sample back to its
state on entry to the function.
C. Finally, the present arrangement can be maintained, in which case
I think that more consistent error messages should be generated by
mle if that is possible.
Gordon
17 years
gretl 1.7.6 schedule
by Allin Cottrell
I've just put out 1.7.6rc2 as a Windows snapshot. I think we're
very close, and unless some nasties are reported soon I plan to
release 1.7.6 tomorrow.
So if anyone has translation updates they'd like to get into
1.7.6, please commit them.
Allin.
17 years
gretl functions and lists (long)
by Allin Cottrell
I'd like to put to people an issue that Jack and I have been
discussing lately, and that seems to require a
backward-incompatible change at some point.
By way of background, recall that user-defined functions in gretl
are supposed to respect the principle of encapsulation.
Basically, this means that
* external variables are accessible within a function only insofar
as they are passed as arguments; and
* no variables are changed at the caller level other than via
assignment of a return value offered by a function.
In addition, settings of program parameters using the "set"
command, within a function, are confined to that function: we save
the values at the caller level and restore them on exit from the
function. And when a model is estimated within a function, this
does not displace the "last model" (the target for accessors such
as $coeff) from the point of view of the caller: this will be the
last model estimated before the function was called.
So far so good, but the trouble is that the current handling of
lists, both as arguments to functions and as return values, is not
wholly consistent with encapsulation. Sven noted one aspect of
this a while back; Jack noticed another aspect recently.
(1) Sven's point: Suppose a function constructs and returns a list
of variables. As a toy example, suppose it returns a list holding
the cubes of the variables given in a list argument. As a
function writer, one might construct the names of the variables in
the return list as <varname_i>_3, where <varname_i> represents the
name of the variable at position i in the input list.
The issue is this: it's true that nothing will be changed at the
caller level unless this return value is assigned, but all the
same such assignment may have an unintended effect. If there
already exists, say, a variable named "x_3", and if a variable "x"
is passed to the cube function, then x_3 will be overwritten by
the cube of x.
I would argue that this is not an insurmountable problem. I think
we can avoid excessive trouble by ensuring that any public gretl
function packages that do this sort of thing are very explicit in
their documentation. We could add a required field to gretl
function packages that return lists -- a warning along the lines
of "This function returns a list of variables named on the pattern
<pattern>. Please note that any existing variables of the same
name will be overwritten."
(2) Jack's point: Consider a function that accepts a list as an
argument, and suppose that the function is called, with a list
argument that includes a series named "x".
Now let's say the writer of this function wants to create a local
scalar called "x". What happens? You'll get a type error, from
trying to overwrite a series with a scalar. There can be only one
variable named "x" inside the function, and the series x is made
"visible" within the function by virtue of its inclusion in a list
argument.
And what happens if the function writer tries to create a local
series named "x"? If the list argument is marked "const" you'll
get an error; if not, the external "x" will be silently
over-written. Bad.
A moment's thought shows that this problem is less tractable than
the first: it's a design flaw, and it can't be solved simply by
documenting clearly what the function does.
Here's a proposal for solving the second problem. It is
implemented in CVS but not by default, only if you edit
lib/src/gretl_func.c and change the definition
#define PROTECT_LISTS 0
to
#define PROTECT_LISTS 1
The essential component is that, when a series is passed to a
function as part of a list argument, the series is _not_
visible/accessible by name within the function. In relation to
the problem cases mentioned above, it will then be OK for the
function writer to create a local scalar or series "x": this won't
collide with a list-argument series "x" since the latter will
never by "seen" by name within the function.
Accordingly, we modify the action of a "foreach i" loop over the
members of a list argument to a function. Instead of "$i"
retrieving the name of the series at position i in the list, it
retrieves the ID number of that series (and series ID numbers are
always unique, regardless of where we are in terms of function
execution).
Two questions are likely to arise here:
1. How do you construct a new series name based on the name of a
series given via a list argument (as in the cubes example)?
Answer: You can get the input variable's name as a string variable
using the built-in varname() function, with the variable's ID
number as argument.
2. How do you get hold of the values of a series given via a list
argument, for use on the right-hand side of a genr expression?
Answer: Use the new built-in function varcopy(). Again, this
takes a variable's ID number as argument, circumventing the
possible collision of names.
Illustration: Take the trivial cubes example -- here is a
new-style implementation
function make_cubes (list xlist)
list retlist = null
loop foreach i xlist
string oldname = varname($i)
sprintf newname "%14s_3", oldname
series @newname = varcopy($i)^3
setinfo @newname -d "cube of @oldname"
list retlist += @newname
end loop
return list retlist
end function
Two questions for you: First, does this solution seem acceptable?
Second, if it's OK, how soon should we aim to make the change?
Allin.
17 years
Function packages & negation operator
by Gordon Hughes
A couple of points. [For reference I am using 1.7.6rc1 dated 27-7-08
on Windows & Linux.]
A. I have been trying to upload updated versions of my SFA packages
to the gretl server. The packages are created and seem to operate
properly on my computer. But the upload generates an error message:
"couldn't parse current_fnfiles/sfa_mod.gfn" where sfa_mod.gfn is the
package name. I have had the same error under both Windows and Linux.
B. The behaviour of the package editor suggests that a sample script
is required. Is this correct? As I recall this was not the case in
the past, but now the save command generates an error message if
there is no sample script. I understand that it may be good practice
but should it be a requirement? Also, it would be convenient if the
button marked "Edit sample script" could provide the option to load a
pre-written script.
Finally, going back to the problem that appeared to affect the
negation operator. Unfortunately, by the time I read the request to
send a copy of the problem script I had edited my script files so
much that I cannot recover a version that generated the error. I
will see if I can reproduce the problem but it may have been a one off.
Gordon
17 years
Function packages - addendum
by Gordon Hughes
On further investigation, after closing gretl and starting again,
when I go File/Function files/On local machine I get the following
error message - "XmlParseFile failed on ...\sfa_mod.gfn". So it
seems that the problem lies in the creation of the xml code for gfn
function package.
Gordon
17 years