8:13 a.m.
Hi,
I'm trying to use the regls() function (from the regls addon) inside my
own hansl function. In my function I'm creating a list "Xw" which I then
pass as regressors to the called regls().
In this situation I'm getting an error message from regls's internal
function outer_series_id -- for convenience, here's what the function
looks like:
<hansl-regls>
function list outer_series_id (const string arg)
list L
loop i=1..$nvars
if varname(i) == arg
L = {i}
break
endif
endloop
return L
end function
</hansl-regls>
The error message says:
"'if'-Auswertungsfehler
if ak
*** error within loop in function outer_series_id
if varname(i) == arg
called by function regls
*** error in function regls, line 193
"
I think I've diagnosed the problem to be that $nvars is much much larger
than the number of elements in the list Xw (+1 for the dependent
variable), and that varname(i) at some point throws an error because of
that.
If I replace the if-line above with the following two lines, then
everything seemingly runs OK:
catch string vname = varname(i)
if !$error && vname == arg
I'm not sure if the use of $nvars in outer_series_id is a bug on the
regls side, or if there's a bug with gretl concerning which sub-parts of
the dataset (which series) are available in a nested function call here.
This is with gretl 2022a (no snapshot) on Windows.
cheers
sven
1:01 p.m.
Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
I think I've diagnosed the problem to be that $nvars is much much
larger
than the number of elements in the list Xw (+1 for the dependent
variable), and that varname(i) at some point throws an error because of
that.
If I replace the if-line above with the following two lines, then
everything seemingly runs OK:
catch string vname = varname(i)
if !$error && vname == arg
Thanks, Allin, I saw that you put this fix already in the latest snapshot.
I'm not sure if the use of $nvars in outer_series_id is a bug on
the
regls side, or if there's a bug with gretl concerning which sub-parts of
the dataset (which series) are available in a nested function call here.
OK, so let me get this straight:
- $nvars is fine, because it is documented to refer to the whole (outer)
dataset, it is not confined to any function scope.
- What's a bit surprising is that varname() also has access to the
outermost scope, so calling varname(1) inside a (user-defined hansl)
function actually gives you the name of the first variable in the
dataset, even if no data was passed to the function. To me this seems to
go a bit against the spirit of hansl functions. I'm not saying that it
should necessarily be changed, but I guess it should be mentioned more
explicitly in the documentation.
- What you're doing with the outer_series_id() function in regls
basically seems to be the same thing that the built-in function varnum()
exists for. However, my testing showed that varnum() _is_ strictly
confined to the function scope, and so it cannot be used here. (Inside a
function it returns NA when called with a string arg that corresponds to
a variable name in the outer scope.) But this somehow seems to suggest
that a more elegant solution should exist for this use case, instead of
having to write a 11-line function just for that. Maybe varnum() should
also be enabled for the outer scope just like varname()? Or only
optionally with an extra switch argument?
- [Side remark: The error thrown by varname(i) for i too large is simply
"invalid arg", whereas varnames(deflist(i)) yields the more informative
"out of bounds" error.]
thanks
sven
10:35 a.m.
Hi, one other trivial question about regls:
In the result bundle we have the member "B" with all coefficients, and
"nzb" with the non-zero coefficients. Fine, but then we also have
"nzX".
What I'm saying is that we have a mixture of upper- and lowercase
conventions here. Would it be too late to introduce "nzB" as an alias
for "nzb"? Or maybe even better to have "b" mean the same thing as
"B",
since the beta vector is also a small greek letter?
Of course this is not important, but it caught my attention.
thanks
sven
3:59 p.m.
On Fri, May 20, 2022 at 10:35 AM Sven Schreiber <svetosch(a)gmx.net> wrote:
Hi, one other trivial question about regls:
In the result bundle we have the member "B" with all coefficients, and
"nzb" with the non-zero coefficients. Fine, but then we also have
"nzX".
What I'm saying is that we have a mixture of upper- and lowercase
conventions here. Would it be too late to introduce "nzB" as an alias
for "nzb"? Or maybe even better to have "b" mean the same thing as
"B",
since the beta vector is also a small greek letter?
I see your point, but here's my rationale for the use of 'b' versus
'B'. Other than in the (unlikely, not very practical) case where the
user wants results for a single penalty value, the object "B" in the
bundle returned by regls will be a matrix with several columns, while
nzb will always be a column vector holding the non-zero elements of
the "optimal" coeff vector (where "optimality" is determined either
via BIC or cross validation).
Allin
4:11 p.m.
Am 20.05.2022 um 21:59 schrieb Cottrell, Allin:
On Fri, May 20, 2022 at 10:35 AM Sven Schreiber
<svetosch(a)gmx.net> wrote:
>
> Hi, one other trivial question about regls:
>
> In the result bundle we have the member "B" with all coefficients, and
> "nzb" with the non-zero coefficients. Fine, but then we also have
"nzX".
> What I'm saying is that we have a mixture of upper- and lowercase
> conventions here. Would it be too late to introduce "nzB" as an alias
> for "nzb"? Or maybe even better to have "b" mean the same thing
as "B",
> since the beta vector is also a small greek letter?
I see your point, but here's my rationale for the use of 'b' versus
'B'. Other than in the (unlikely, not very practical) case where the
user wants results for a single penalty value, the object "B" in the
bundle returned by regls will be a matrix with several columns, while
nzb will always be a column vector holding the non-zero elements of
the "optimal" coeff vector (where "optimality" is determined either
via BIC or cross validation).
Right, that makes sense - except in the case where single_b is switched
on and a single output vector is requested for B.
But actually this makes me think: Could there be a case for a matrices
array nzB of length nlambda, where each entry holds the associated
nzb-s? And perhaps likewise nzLX an array of lists with individual nzX-s
in them? But maybe it's overkill.
thanks
sven
5:10 p.m.
On Fri, May 20, 2022 at 4:11 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 20.05.2022 um 21:59 schrieb Cottrell, Allin:
> On Fri, May 20, 2022 at 10:35 AM Sven Schreiber <svetosch(a)gmx.net> wrote:
>>
>> Hi, one other trivial question about regls:
>>
>> In the result bundle we have the member "B" with all coefficients,
and
>> "nzb" with the non-zero coefficients. Fine, but then we also have
"nzX".
>> What I'm saying is that we have a mixture of upper- and lowercase
>> conventions here. Would it be too late to introduce "nzB" as an alias
>> for "nzb"? Or maybe even better to have "b" mean the same
thing as "B",
>> since the beta vector is also a small greek letter?
>
> I see your point, but here's my rationale for the use of 'b' versus
> 'B'. Other than in the (unlikely, not very practical) case where the
> user wants results for a single penalty value, the object "B" in the
> bundle returned by regls will be a matrix with several columns, while
> nzb will always be a column vector holding the non-zero elements of
> the "optimal" coeff vector (where "optimality" is determined
either
> via BIC or cross validation).
Right, that makes sense - except in the case where single_b is switched
on and a single output vector is requested for B.
But actually this makes me think: Could there be a case for a matrices
array nzB of length nlambda, where each entry holds the associated
nzb-s? And perhaps likewise nzLX an array of lists with individual nzX-s
in them? But maybe it's overkill.
IMO it would indeed be overkill. You have the big B matrix if you want
to do anything non-standard. Cutting a given column of B down to its
non-zero elements is easy enough given the tools available in hansl,
but only offers a marginal efficiency gain relative to using the
column "as-is", with zeros included.
Allin
2:53 p.m.
New subject: regls and MPI (was Re: Internal error from the regls addon)
Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
Hi,
I'm trying to use the regls() function (from the regls addon) ...
OK, carrying on exploring the regls world, sorry for the many emails; but:
The cross-validation result output table mentioned in the documentation
p. 4 (with column headings s, MSE, se) is not shown here for me in the
case of the sample script, for example. When I switch off MPI, then I
can see it.
This might be Windows-specific, because in the past grabbing the MPI
output turned out to be difficult sometimes.
thanks
sven
2:21 p.m.
New subject: regls and MPI (was Re: Internal error from the regls addon)
On Fri, May 20, 2022 at 2:53 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
> Hi,
> I'm trying to use the regls() function (from the regls addon) ...
OK, carrying on exploring the regls world, sorry for the many emails; but:
The cross-validation result output table mentioned in the documentation
p. 4 (with column headings s, MSE, se) is not shown here for me in the
case of the sample script, for example. When I switch off MPI, then I
can see it.
This might be Windows-specific, because in the past grabbing the MPI
output turned out to be difficult sometimes.
Thanks for the report. Yes, this is Windows-specific, and at present I
suspect it's due to a GLib bug. I plan to experiment with an update to
GLib.
We have two ways of getting output back from MPI into the GUI: a
"realtime" method that grabs chunks of output as they become
available, and a "wait till it's done" method. The first one works
nicely on Linux but I haven't managed to get it working on Windows.
Today I tried the regls example on both platforms using the second
method -- calling the same GLib function in both cases. I got all the
output on Linux but not on Windows. Hence my thought that GLib (in the
version we've been packaging for Windows) may not be right.
Allin
4:08 p.m.
New subject: regls and MPI (was Re: Internal error from the regls addon)
On Sun, May 22, 2022 at 2:21 PM Cottrell, Allin <cottrell(a)wfu.edu> wrote:
On Fri, May 20, 2022 at 2:53 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
>
> Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
> > Hi,
> > I'm trying to use the regls() function (from the regls addon) ...
>
> OK, carrying on exploring the regls world, sorry for the many emails; but:
>
> The cross-validation result output table mentioned in the documentation
> p. 4 (with column headings s, MSE, se) is not shown here for me in the
> case of the sample script, for example. When I switch off MPI, then I
> can see it.
> This might be Windows-specific, because in the past grabbing the MPI
> output turned out to be difficult sometimes.
Thanks for the report. Yes, this is Windows-specific, and at present I
suspect it's due to a GLib bug. I plan to experiment with an update to
GLib.
Updating Glib didn't help on that point, but I think I've now fixed
the issue using the win32 CreateProcess API directly.
Allin
12:24 p.m.
New subject: regls and MPI (was Re: Internal error from the regls addon)
Am 23.05.2022 um 22:08 schrieb Cottrell, Allin:
On Sun, May 22, 2022 at 2:21 PM Cottrell, Allin
<cottrell(a)wfu.edu> wrote:
>
> On Fri, May 20, 2022 at 2:53 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
>>
>> Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
>>> Hi,
>>> I'm trying to use the regls() function (from the regls addon) ...
>>
>> OK, carrying on exploring the regls world, sorry for the many emails; but:
>>
>> The cross-validation result output table mentioned in the documentation
>> p. 4 (with column headings s, MSE, se) is not shown here for me in the
>> case of the sample script, for example. When I switch off MPI, then I
>> can see it.
>> This might be Windows-specific, because in the past grabbing the MPI
>> output turned out to be difficult sometimes.
>
> Thanks for the report. Yes, this is Windows-specific, and at present I
> suspect it's due to a GLib bug. I plan to experiment with an update to
> GLib.
Updating Glib didn't help on that point, but I think I've now fixed
the issue using the win32 CreateProcess API directly.
Indeed, with yesterday's snapshot the regls sample script now shows me
the s/MSE/se table alright! (with MPI)
thanks, Allin
Sven
3:17 p.m.
New subject: regls nz_print problem (was Re: Internal error from the regls addon)
Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
Hi,
I'm trying to use the regls() function
... there's one more: I think regls's internal nz_print function has a
small bug in the line "len = max(strlen(S))". The problem is that S and
by implication strlen(S) may be empty, and then max() throws an error.
(I got an error from that in a corner case.)
I would suggest to enclose the entire printing at the end of the
function with a check like "if nelem(S)".
thanks
sven
7:08 p.m.
New subject: regls nz_print problem (was Re: Internal error from the regls addon)
On Fri, May 20, 2022 at 3:17 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
> Hi,
> I'm trying to use the regls() function
... there's one more: I think regls's internal nz_print function has a
small bug in the line "len = max(strlen(S))". The problem is that S and
by implication strlen(S) may be empty, and then max() throws an error.
(I got an error from that in a corner case.)
I would suggest to enclose the entire printing at the end of the
function with a check like "if nelem(S)".
I'll look into it, but could you give me an idea of what the "corner
case" involved?
Allin
4:57 a.m.
New subject: regls nz_print problem (was Re: Internal error from the regls addon)
Am 21.05.2022 um 01:08 schrieb Cottrell, Allin:
On Fri, May 20, 2022 at 3:17 PM Sven Schreiber
<svetosch(a)gmx.net> wrote:
>
> Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
>> Hi,
>> I'm trying to use the regls() function
>
> ... there's one more: I think regls's internal nz_print function has a
> small bug in the line "len = max(strlen(S))". The problem is that S and
> by implication strlen(S) may be empty, and then max() throws an error.
> (I got an error from that in a corner case.)
> I would suggest to enclose the entire printing at the end of the
> function with a check like "if nelem(S)".
I'll look into it, but could you give me an idea of what the "corner
case" involved?
Well, I think it happens when no regressors are left in the model. This
could be because a certain single lambda is imposed, or simply because
the regressors are really bad predictors such that the best model
doesn't need them.
cheers
sven
2:22 p.m.
New subject: regls nz_print problem (was Re: Internal error from the regls addon)
On Sat, May 21, 2022 at 4:57 AM Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 21.05.2022 um 01:08 schrieb Cottrell, Allin:
> On Fri, May 20, 2022 at 3:17 PM Sven Schreiber <svetosch(a)gmx.net> wrote:
>>
>> Am 18.05.2022 um 14:13 schrieb Sven Schreiber:
>>> Hi,
>>> I'm trying to use the regls() function
>>
>> ... there's one more: I think regls's internal nz_print function has a
>> small bug in the line "len = max(strlen(S))". The problem is that S
and
>> by implication strlen(S) may be empty, and then max() throws an error.
>> (I got an error from that in a corner case.)
>> I would suggest to enclose the entire printing at the end of the
>> function with a check like "if nelem(S)".
>
> I'll look into it, but could you give me an idea of what the "corner
> case" involved?
Well, I think it happens when no regressors are left in the model. This
could be because a certain single lambda is imposed, or simply because
the regressors are really bad predictors such that the best model
doesn't need them.
OK, that should now be fixed in version 0.31 of regls.
Allin
864
days inactive
870
days old
13 comments
2 participants
participants (2)
-
Cottrell, Allin -
Sven Schreiber