4:55 a.m.
Hi,
there seems to be a new bug introduced after 1.7.4; as always, I can
provide more details later and/or file a bug report, but maybe this is
already enough, and before I forget about it, here goes:
In a script of ours, we have an "omit" test statement after a standard
(OLS) estimation, but in between there are some other calculations
("scalar" and "matrix", so effectively "genr" I guess).
This worked fine up to 1.7.4, but current snapshots complain at the
"omit" line, saying that no estimate has been performed so far.
cheers,
sven
9:06 a.m.
On Thu, 22 May 2008, Sven Schreiber wrote:
there seems to be a new bug introduced after 1.7.4; as always, I
can provide more details later and/or file a bug report, but
maybe this is already enough, and before I forget about it, here
goes:
In a script of ours, we have an "omit" test statement after a
standard (OLS) estimation, but in between there are some other
calculations ("scalar" and "matrix", so effectively "genr"
I
guess).
Please send a test case, I can't replicate this. For example, the
following works fine:
<script>
open data9-7
list xlist = 0 2 3 4
# see 'omit' at end
ols 1 xlist
matrix X = {xlist}
matrix XTXinv = inv(X'X)
scalar s2 = $sigma * $sigma
matrix V = s2*XTXinv
matrix se = sqrt(diag(V))
printf "\n%.12E\n", se
# 'omit' matches ols above
omit 3 4
</script>
Allin.
8:57 a.m.
Am 22.05.2008 15:06, Allin Cottrell schrieb:
On Thu, 22 May 2008, Sven Schreiber wrote:
> there seems to be a new bug introduced after 1.7.4; as always, I
> can provide more details later and/or file a bug report, but
> maybe this is already enough, and before I forget about it, here
> goes:
>
> In a script of ours, we have an "omit" test statement after a
> standard (OLS) estimation, but in between there are some other
> calculations ("scalar" and "matrix", so effectively
"genr" I
> guess).
Please send a test case, I can't replicate this. For example, the
following works fine:
Ok, it seems to be related to calling a user-defined function after
estimation and before the restriction test; here is a demo script:
<script>
open denmark
function trivial(void)
result = 1
return scalar result
end function
ols LRM 0 LRY
scalar check = trivial()
omit LRY
</script>
This gives the mentioned error on today's CVS. I can't guarantee right
now that this demo script works with 1.7.4, but our real-world script
(where we noticed the bug) did.
In this case there is obviously an easy workaround (move the function
call after the omit), but I can imagine cases where you want to use
'restrict' with some value that you need to calculate in a function
beforehand. So I'm inclined to think it's a bug.
thanks,
sven
2:50 p.m.
On Fri, 23 May 2008, Sven Schreiber wrote:
Am 22.05.2008 15:06, Allin Cottrell schrieb:
> On Thu, 22 May 2008, Sven Schreiber wrote:
>
> > In a script of ours, we have an "omit" test statement after
> > a standard (OLS) estimation, but in between there are some
> > other calculations ("scalar" and "matrix", so effectively
> > "genr" I guess).
Ok, it seems to be related to calling a user-defined function
after estimation and before the restriction test; here is a demo
script:
<script>
open denmark
function trivial(void)
result = 1
return scalar result
end function
ols LRM 0 LRY
scalar check = trivial()
omit LRY
</script>
Thanks, I now see (and remember!) what's happening. As you know,
"omit" works on the "last model". Sometime after the 1.7.4
release I noticed that we had a problem with functions: if a
function estimated a model, this would remain the "last model" on
exit from the function, which is not what we want.
I then "fixed" this by simply NULLing out the last model on exit
from a function, with the results you found.
I've now implemented the right fix, namely, when we enter a
function we save a pointer to the current "last model" and on exit
from a function we restore that pointer.
Should be OK in CVS and Windows snapshot.
Allin.
4:27 p.m.
Am 23.05.2008 20:50, Allin Cottrell schrieb:
I've now implemented the right fix, namely, when we enter a
function we save a pointer to the current "last model" and on exit
from a function we restore that pointer.
Should be OK in CVS and Windows snapshot.
Yes it looks like it, thanks.
However, another new problem just appeared, the context is as follows:
First we do IV estimation:
tsls rpivstand endorhs d_predrhs ; d_predrhs d1_wox --robust
Then we do various stuff:
accessing $coeff, $vcv>
<some 'printf'-ing,
'scalar', 'matrix' operations>
<a function call -- see the previous bug>
<some 'omit' tests>
<some 'restrict' tests>
and then we access the hausman test result and calculate a p-value by
assuming one d.o.f. less than usual (don't ask why right now):
matrix hausmanresult = $hausman
pvalue X hausmanresult[1,2]-1 hausmanresult[1,1]
-- and this now gives an "invalid argument" error (it didn't in 1.7.4).
Note that the following reformulation of the last line provides a
workaround, so it seems it has to do with "on-the-fly calculations" (if
that's the right term):
temp=hausmanresult[1,2]-1
pvalue X temp hausmanresult[1,1]
BTW, on various occasions I noticed that arithmetic expressions cannot
always be used in place of simple variables. But sometimes it does seem
to work. Is there some rule when it should work, or is it sheer luck
when it does?
Thanks,
Sven
4:44 p.m.
On Fri, 23 May 2008, Sven Schreiber wrote:
Am 23.05.2008 20:50, Allin Cottrell schrieb:
> Should be OK in CVS and Windows snapshot.
>
Yes it looks like it, thanks.
Good.
However, another new problem just appeared...
matrix hausmanresult = $hausman
pvalue X hausmanresult[1,2]-1 hausmanresult[1,1]
-- and this now gives an "invalid argument" error (it didn't in
1.7.4)...
Oof, this is to do with a change I introduced at Stefano's
behest: not "casting" 1x1 matrix results to scalars automatically.
I'll have to think about that some more.
BTW, on various occasions I noticed that arithmetic expressions
cannot always be used in place of simple variables. But
sometimes it does seem to work. Is there some rule when it
should work, or is it sheer luck when it does?
In general, arbitrary arithmetic should work for arguments to
built-in functions (or user-defined functions for that matter),
while it's not very likely to work for arguments to straight
"commands". However, in this particular case using the pvalue
function won't help, right now, since it's a matrix/scalar thing.
I'll try to fix this properly for 1.7.5.
Allin.
5:34 p.m.
On Fri, 23 May 2008, Allin Cottrell wrote:
On Fri, 23 May 2008, Sven Schreiber wrote:
> matrix hausmanresult = $hausman
> pvalue X hausmanresult[1,2]-1 hausmanresult[1,1]
>
> -- and this now gives an "invalid argument" error (it didn't in
> 1.7.4)...
Oof, this is to do with a change I introduced at Stefano's
behest: not "casting" 1x1 matrix results to scalars
automatically. I'll have to think about that some more.
In CVS and the snapshot there's a fix which takes care of all the
probability-related functions (including pvalue). I'll search
some more later for other functions that might be affected.
Allin.
5:53 p.m.
On Fri, 23 May 2008, Allin Cottrell wrote:
Oof, this is to do with a change I introduced at Stefano's
behest: not "casting" 1x1 matrix results to scalars automatically.
I'll have to think about that some more.
I can see Stefano's point, but there are also advantages in auto-casting a
1x1 matrix to a scalar. By the way (Stefano, are you there?) if you really
need to ensure that the result of an operation is a matrix, you can always
force it via the "matrix" alias, as in
matrix x = foo(z)
So, in my view, there are pros and cons to either choice. I think we ought
to reach a collective decision here. So my proposal is: everyone
interested, please have your say now. (Myself, I'd say "auto-cast".)
Allin, just wait for a few days and then whatever you decide is the law.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
11 a.m.
Am 23.05.2008 22:44, Allin Cottrell schrieb:
On Fri, 23 May 2008, Sven Schreiber wrote:
> However, another new problem just appeared...
>
> matrix hausmanresult = $hausman
> pvalue X hausmanresult[1,2]-1 hausmanresult[1,1]
>
> -- and this now gives an "invalid argument" error (it didn't in
> 1.7.4)...
Oof, this is to do with a change I introduced at Stefano's
behest: not "casting" 1x1 matrix results to scalars automatically.
I'll have to think about that some more.
first, one thing I don't understand here: the second term
(hausmanresult[1,1]) doesn't seem problematic (see the workaround I
mentioned, where it is unchanged). But that would mean it is treated as
a scalar, no? Then why does the first term (hausmanresult[1,2]) give
something different?
secondly, I think there are two different issues involved here:
1) The first one is what data type is returned by a function (built-in
or user-defined).
2) The second issue is what happens if you pick elements from a matrix,
as in the above example.
From my experience with earlier discussions about those issues on the
numpy (numerical python) mailing list, these issues can and should be
decided separately.
With respect to the first issue, I have some sympathy with Stefano's
view that the returned datatype should not depend on the specific matrix
dimension in particular cases.
As for the second issue, I would say that explicitly indexing a single
scalar element from a matrix --as in the above example-- should return a
scalar. However, that would *not* mean that each indexing range which
refers to a 1x1 submatrix should be returned as a scalar. For example,
let's say somebody picks the first row of a matrix mymat by using
mymat[1,] in an expression. In general the first row would not be just a
single element, and as such mymat[1,] should always be a matrix, even if
it turns out to be just a 1x1 matrix.
[and now continuing a different topic:]
> BTW, on various occasions I noticed that arithmetic expressions
> cannot always be used in place of simple variables. But
> sometimes it does seem to work. Is there some rule when it
> should work, or is it sheer luck when it does?
In general, arbitrary arithmetic should work for arguments to
built-in functions (or user-defined functions for that matter),
while it's not very likely to work for arguments to straight
"commands".
In principle I think I understand, but I'm not sure the demarcation
between commands and functions is very clear in gretl. What would that
'pvalue' thing be for example?
-sven
3:15 p.m.
On Sat, 24 May 2008, Sven Schreiber wrote:
[ first, about cating 1x1 matrices to scalars, to which I'll reply
later, then... ]
> > BTW, on various occasions I noticed that arithmetic
> > expressions cannot always be used in place of simple
> > variables. But sometimes it does seem to work. Is there some
> > rule when it should work, or is it sheer luck when it does?
>
> In general, arbitrary arithmetic should work for arguments to
> built-in functions (or user-defined functions for that
> matter), while it's not very likely to work for arguments to
> straight "commands".
In principle I think I understand, but I'm not sure the
demarcation between commands and functions is very clear in
gretl. What would that 'pvalue' thing be for example?
For the most part the distinction is pretty clear: a function is
something where all the arguments are wrapped in parentheses and
separated by commas, and a command is everything else. But there
are a few hybrid cases, and "pvalue" is one such. Since we have a
perfectly good pvalue function with the same semantics, we
implement the "command" form by composing and executing a call to
"genr"; this gives more flexibility on the form of the arguments.
Allin.
2:32 p.m.
On Fri, 30 May 2008, Sven Schreiber wrote:
Allin Cottrell schrieb:
> On Sat, 24 May 2008, Sven Schreiber wrote:
>
> [ first, about cating 1x1 matrices to scalars, to which I'll reply later,
> then... ]
>
any news here about the 1x1 matrix treatment?
The current status is:
* If a genr expression "naturally" produces a matrix result, but
the matrix is 1x1, and the user has not forced the issue by
specifying a result-type of "scalar" or "matrix", then we leave
the result as a 1x1 matrix. (As per Stefano's suggestion.)
* But we've tried to make inter-conversion between 1x1 matrices
and scalars (inside genr) more flexible, so that the type of
the result should make less difference to the user.
Allin.
6011
days inactive
6019
days old
11 comments
3 participants
participants (3)
-
Allin Cottrell -
Riccardo (Jack) Lucchetti -
Sven Schreiber