7:24 a.m.
Hi,
the doc for nadarwat says: "a popular choice is n^-0.2". Why not make
that the default value officially, such that the third/final arg becomes
optional?
thanks,
sven
7:30 a.m.
Am 08.12.2017 um 13:24 schrieb Sven Schreiber:
the doc for nadarwat says: "a popular choice is n^-0.2".
Why not make
that the default value officially, such that the third/final arg becomes
optional?
Sorry, a P.S. question about nadarwat() and also the other nonparametric
estimator "loess()": Why are both only implemented as functions instead
of commands? For estimators I would have expected a command.
Thanks,
sven
6:01 p.m.
On Fri, 8 Dec 2017, Sven Schreiber wrote:
Am 08.12.2017 um 13:24 schrieb Sven Schreiber:
> the doc for nadarwat says: "a popular choice is n^-0.2". Why not
> make that the default value officially, such that the third/final
> arg becomes optional?
Sorry, a P.S. question about nadarwat() and also the other
nonparametric estimator "loess()": Why are both only implemented
as functions instead of commands? For estimators I would have
expected a command.
For any parametric estimator, yes, a command would be expected. With
parametric estimation, predicted values may be of interest but in
the first instance interest centers on the parameter estimates, and
the user naturally wants to see them (along with standard errors,
etc.).
But with loess and nadarwat (and also svmpred) there's nothing of
much interest _other than_ the predicted values. Commands don't
return anything, so instead we implement these as functions that
return the predictions series.
If we implemented the non-parametric estimation functions as
commands, what (if anything) would we print? We could print the
predictions, I suppose, but that's unlikely to be of much use; the
user would need an accessor to actually "get hold of" them. So we
return the predictions instead.
Allin
3:50 a.m.
Am 09.12.2017 um 00:01 schrieb Allin Cottrell:
On Fri, 8 Dec 2017, Sven Schreiber wrote:
> Sorry, a P.S. question about nadarwat() and also the other
> nonparametric estimator "loess()": Why are both only implemented as
> functions instead of commands? For estimators I would have expected a
> command.
For any parametric estimator, yes, a command would be expected. With
parametric estimation, predicted values may be of interest but in the
first instance interest centers on the parameter estimates, and the user
naturally wants to see them (along with standard errors, etc.).
But with loess and nadarwat (and also svmpred) there's nothing of much
interest _other than_ the predicted values. Commands don't return
anything, so instead we implement these as functions that return the
predictions series.
If we implemented the non-parametric estimation functions as commands,
what (if anything) would we print? We could print the predictions, I
suppose, but that's unlikely to be of much use; the user would need an
accessor to actually "get hold of" them. So we return the predictions
instead.
Yes that makes sense. OTOH for estimation commands we now have the
$model bundle which also holds meta stuff like $T and so on (I think).
Perhaps it could be useful to extend that to the non-parametric
estimators as well. Something like an optional bundle-pointer argument.
But these are just thoughts about design principles, I don't currently
have an explicit use case for it.
cheers,
sven
6:03 a.m.
Am 08.12.2017 um 13:24 schrieb Sven Schreiber:
the doc for nadarwat says: "a popular choice is n^-0.2". Why not make
that the default value officially, such that the third/final arg
becomes optional?
I think this was never resolved, because the thread got distracted (by
myself). So what about it?
thanks,
sven
9:48 a.m.
But why not Silverman’s rule-of-thumb estimator (1.06*sd(X)*n^-0.2)? The
optimal bandwidth *should* be of *order* n^-0.2, but should also depend on
the spread of X! The exact formula is (R(K)/(\sigma^2_K*R(f'')))^-0.2 *
n^-0.2, where R(f'') depends on the actual density and therefore the
distribution of X’s, and therefore, its spread.
If I have a dataset, say, with length/weight data, it would be very silly
to use the same bandwidth for length in metres and for length in
millimetres because it would over-smooth in the first case and under-smooth
in the second!
For reference, see Scott (2015) “Multivariate estimation”, 2e, p. 144,
“Normal reference rule”.
R uses this rule by default (bw.nrd). In fact, it guards against outliers
by using 1.06*min(sd(X), IQR(X)/1.34)*n^-0.2.
Or is anyone in favour of the old rule for the sake of backwards
compatibility?
--
Bien cordialement, | Yours sincerely,
Andreï V. Kostyrka.
http://kostyrka.ru, http://kostyrka.ru/blog
On 23 May 2018 at 12:03, Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 08.12.2017 um 13:24 schrieb Sven Schreiber:
>
> the doc for nadarwat says: "a popular choice is n^-0.2". Why not make
> that the default value officially, such that the third/final arg becomes
> optional?
>
>
I think this was never resolved, because the thread got distracted (by
myself). So what about it?
thanks,
sven
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
10:30 a.m.
Am 23.05.2018 um 15:48 schrieb Andreï V. Kostyrka:
But why not Silverman’s rule-of-thumb estimator (1.06*sd(X)*n^-0.2)?
The optimal bandwidth *should* be of *order* n^-0.2, but should also
depend on the spread of X! The exact formula is
(R(K)/(\sigma^2_K*R(f'')))^-0.2 * n^-0.2, where R(f'') depends on the
actual density and therefore the distribution of X’s, and therefore,
its spread.
If I have a dataset, say, with length/weight data, it would be very
silly to use the same bandwidth for length in metres and for length in
millimetres because it would over-smooth in the first case and
under-smooth in the second!
For reference, see Scott (2015) “Multivariate estimation”, 2e, p. 144,
“Normal reference rule”.
R uses this rule by default (bw.nrd). In fact, it guards against
outliers by using 1.06*min(sd(X), IQR(X)/1.34)*n^-0.2.
What you're saying makes sense I think. (I haven't used this estimator
myself yet.)
Or is anyone in favour of the old rule for the sake of backwards
compatibility?
Fortunately there is no compatibility issue here, because there never
has been a default. So far it was just a remark in the docs.
So should we go with this default?
thanks,
sven
11:06 a.m.
On Wed, 23 May 2018, Sven Schreiber wrote:
Am 23.05.2018 um 15:48 schrieb Andreï V. Kostyrka:
> But why not Silverman’s rule-of-thumb estimator (1.06*sd(X)*n^-0.2)? The
> optimal bandwidth *should* be of *order* n^-0.2, but should also depend on
> the spread of X! The exact formula is (R(K)/(\sigma^2_K*R(f'')))^-0.2 *
> n^-0.2, where R(f'') depends on the actual density and therefore the
> distribution of X’s, and therefore, its spread.
> If I have a dataset, say, with length/weight data, it would be very silly
> to use the same bandwidth for length in metres and for length in
> millimetres because it would over-smooth in the first case and under-smooth
> in the second!
> For reference, see Scott (2015) “Multivariate estimation”, 2e, p. 144,
> “Normal reference rule”.
> R uses this rule by default (bw.nrd). In fact, it guards against outliers
> by using 1.06*min(sd(X), IQR(X)/1.34)*n^-0.2.
What you're saying makes sense I think. (I haven't used this estimator myself
yet.)
> Or is anyone in favour of the old rule for the sake of backwards
> compatibility?
Fortunately there is no compatibility issue here, because there never has
been a default. So far it was just a remark in the docs.
So should we go with this default?
I like this.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:10 a.m.
On Wed, 23 May 2018, Riccardo (Jack) Lucchetti wrote:
> So should we go with this default?
I like this.
Hm, hold on a second.
We also use the bandwidth argument to flag if we want the leave-one-out
estimator or not (via its sign).
I'm afraid the best way to handle this would be introducing a
backward-incompatible change. I can't see an easy way out of this.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:15 a.m.
Am 23.05.2018 um 17:10 schrieb Riccardo (Jack) Lucchetti:
On Wed, 23 May 2018, Riccardo (Jack) Lucchetti wrote:
>> So should we go with this default?
>
> I like this.
>
Hm, hold on a second.
We also use the bandwidth argument to flag if we want the
leave-one-out estimator or not (via its sign).
I'm afraid the best way to handle this would be introducing a
backward-incompatible change. I can't see an easy way out of this.
Well, the default then would not work for this leave-one-out thing,
requiring an explicit input, but that's it. I don't know how popular
this leave-out-variant is, would it be so bad? (Not worse than the
status quo, mind you.)
In general the whole issue is not a big thing of course.
thanks,
sven
11:30 a.m.
On Wed, 23 May 2018, Sven Schreiber wrote:
Am 23.05.2018 um 17:10 schrieb Riccardo (Jack) Lucchetti:
> On Wed, 23 May 2018, Riccardo (Jack) Lucchetti wrote:
>
>>> So should we go with this default?
>>
>> I like this.
>>
>
> Hm, hold on a second.
>
> We also use the bandwidth argument to flag if we want the leave-one-out
> estimator or not (via its sign).
>
> I'm afraid the best way to handle this would be introducing a
> backward-incompatible change. I can't see an easy way out of this.
Well, the default then would not work for this leave-one-out thing, requiring
an explicit input, but that's it. I don't know how popular this
leave-out-variant is, would it be so bad? (Not worse than the status quo,
mind you.)
Actually, the leave-one-out variant is often preferable, so if we had to
have a default value for that as well I would like to have it on.
In general the whole issue is not a big thing of course.
Obviously not.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:39 a.m.
Am 23.05.2018 um 17:30 schrieb Riccardo (Jack) Lucchetti:
On Wed, 23 May 2018, Sven Schreiber wrote:
> Am 23.05.2018 um 17:10 schrieb Riccardo (Jack) Lucchetti:
>> On Wed, 23 May 2018, Riccardo (Jack) Lucchetti wrote:
>>
>>>> So should we go with this default?
>>>
>>> I like this.
>>>
>>
>> Hm, hold on a second.
>>
>> We also use the bandwidth argument to flag if we want the
>> leave-one-out estimator or not (via its sign).
>>
>> I'm afraid the best way to handle this would be introducing a
>> backward-incompatible change. I can't see an easy way out of this.
>
> Well, the default then would not work for this leave-one-out thing,
> requiring an explicit input, but that's it. I don't know how popular
> this leave-out-variant is, would it be so bad? (Not worse than the
> status quo, mind you.)
Actually, the leave-one-out variant is often preferable, so if we had
to have a default value for that as well I would like to have it on.
Yeah well, that default is indeed going to be difficult given the
current behavior. But if you could live with the opposite default then
of course a fourth optional boolean switch could be introduced and
everything would be fine. (And the old-style negative input would have
to be translated internally.)
-sven
10:58 a.m.
Am 23.05.2018 um 17:39 schrieb Sven Schreiber:
Am 23.05.2018 um 17:30 schrieb Riccardo (Jack) Lucchetti:
> On Wed, 23 May 2018, Sven Schreiber wrote:
>
>> Am 23.05.2018 um 17:10 schrieb Riccardo (Jack) Lucchetti:
>>> We also use the bandwidth argument to flag if we want the
>>> leave-one-out estimator or not (via its sign).
>>>
>>> I'm afraid the best way to handle this would be introducing a
>>> backward-incompatible change. I can't see an easy way out of this.
>>
>> Well, the default then would not work for this leave-one-out thing,
>> requiring an explicit input, but that's it. I don't know how popular
>> this leave-out-variant is, would it be so bad? (Not worse than the
>> status quo, mind you.)
>
> Actually, the leave-one-out variant is often preferable, so if we had
> to have a default value for that as well I would like to have it on.
Yeah well, that default is indeed going to be difficult given the
current behavior. But if you could live with the opposite default then
of course a fourth optional boolean switch could be introduced and
everything would be fine. (And the old-style negative input would have
to be translated internally.)
This thread is dangling. Should I open a ticket on the tracker for this
bandwidth default thing?
cheers,
sven
6:01 a.m.
On Tue, 5 Jun 2018, Sven Schreiber wrote:
Am 23.05.2018 um 17:39 schrieb Sven Schreiber:
> Am 23.05.2018 um 17:30 schrieb Riccardo (Jack) Lucchetti:
>>>>
>>>> I'm afraid the best way to handle this would be introducing a
>>>> backward-incompatible change. I can't see an easy way out of this.
[...]
This thread is dangling. Should I open a ticket on the tracker for
this
bandwidth default thing?
I've given the matter some thought, and I am convinced that the best
long-run solution is to deprecate the old function and introduce a new
one. Hence, the following proposal: let's have a function with a slightly
different name (ie nadwat()) with 5 arguments, as in
function series nadwat(seties y, series x, scalar bandwidth[auto],
bool leave_one_out[0], scalar trim[4])
The old function can keep working for a version or two, and then we retire
it together with the nadarwat_trim libset variable.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:28 p.m.
Am 13.06.2018 um 12:01 schrieb Riccardo (Jack) Lucchetti:
On Tue, 5 Jun 2018, Sven Schreiber wrote:
> This thread is dangling. Should I open a ticket on the tracker for
> this bandwidth default thing?
I've given the matter some thought, and I am convinced that the best
long-run solution is to deprecate the old function and introduce a new
one. Hence, the following proposal: let's have a function with a
slightly different name (ie nadwat()) with 5 arguments, as in
function series nadwat(seties y, series x, scalar bandwidth[auto],
bool leave_one_out[0], scalar trim[4])
The old function can keep working for a version or two, and then we
retire it together with the nadarwat_trim libset variable.
I don't see why we need to rename and deprecate the nadarwat function to
achieve what you want. In the old version the user always has to give y,
x and h (bandwidth). Plus in the background there is the nadarwat_trim
setting (default 4).
So if we extend the function as you say, then only two things need to be
done internally if the function is called with only the first three args:
1) Check whether bandwidth is negative. If so, set leave_one_out to 1
and bandwidth to abs(bandwidth).
2) Check whether the nadarwat_trim value is != 4. If so, set the trim
arg to that value.
If instead the (new incarnation of) the function is called with 4 or all
5 arguments, then:
1) Check whether bandwidth is negative. If so, raise an error (since
leave_one_out is now responsible for that choice).
2) Always ignore the nadarwat_trim setting, as the usage of the function
with more than 3 args indicates the new version is meant.
So I think backwards compatibility can be achieved quite easily here.
The deprecation would only apply to the nadarwat_trim setting.
cheers,
sven
1:17 a.m.
On Wed, 13 Jun 2018, Sven Schreiber wrote:
I don't see why we need to rename and deprecate the nadarwat
function to
achieve what you want. In the old version the user always has to give y, x
and h (bandwidth). Plus in the background there is the nadarwat_trim setting
(default 4).
So if we extend the function as you say, then only two things need to be done
internally if the function is called with only the first three args:
1) Check whether bandwidth is negative. If so, set leave_one_out to 1 and
bandwidth to abs(bandwidth).
2) Check whether the nadarwat_trim value is != 4. If so, set the trim arg to
that value.
My concern is: how do you get automatic bandwidth selection and the
leave-one-out option together? I guess that, in your scheme, you could
leave the third argument empty and set the fourth one to 1, but it looks
sort of awkward to me. Anyway, it's a possibility.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
2:43 a.m.
Am 14.06.2018 um 07:17 schrieb Riccardo (Jack) Lucchetti:
On Wed, 13 Jun 2018, Sven Schreiber wrote:
> I don't see why we need to rename and deprecate the nadarwat function
> to achieve what you want. In the old version the user always has to
> give y, x and h (bandwidth). Plus in the background there is the
> nadarwat_trim setting (default 4).
>
> So if we extend the function as you say, then only two things need to
> be done internally if the function is called with only the first
> three args:
> 1) Check whether bandwidth is negative. If so, set leave_one_out to 1
> and bandwidth to abs(bandwidth).
> 2) Check whether the nadarwat_trim value is != 4. If so, set the trim
> arg to that value.
My concern is: how do you get automatic bandwidth selection and the
leave-one-out option together? I guess that, in your scheme, you could
leave the third argument empty and set the fourth one to 1, but it
looks sort of awkward to me. Anyway, it's a possibility.
Jack, please reformulate your proposal in strict hansl. Because you wrote:
function series nadwat(seties y, series x, scalar bandwidth[auto],
bool leave_one_out[0], scalar trim[4])
but there is no such thing as an "auto" default value in hansl AFAIK,
and also you put the default for leave_one_out to 0 whereas verbally it
sounds as if you want to have 1. So this is confusing.
thanks,
sven
2:52 a.m.
On Thu, 14 Jun 2018, Sven Schreiber wrote:
Am 14.06.2018 um 07:17 schrieb Riccardo (Jack) Lucchetti:
> On Wed, 13 Jun 2018, Sven Schreiber wrote:
>
>> I don't see why we need to rename and deprecate the nadarwat function to
>> achieve what you want. In the old version the user always has to give y, x
>> and h (bandwidth). Plus in the background there is the nadarwat_trim
>> setting (default 4).
>>
>> So if we extend the function as you say, then only two things need to be
>> done internally if the function is called with only the first three args:
>> 1) Check whether bandwidth is negative. If so, set leave_one_out to 1 and
>> bandwidth to abs(bandwidth).
>> 2) Check whether the nadarwat_trim value is != 4. If so, set the trim arg
>> to that value.
>
> My concern is: how do you get automatic bandwidth selection and the
> leave-one-out option together? I guess that, in your scheme, you could
> leave the third argument empty and set the fourth one to 1, but it looks
> sort of awkward to me. Anyway, it's a possibility.
Jack, please reformulate your proposal in strict hansl. Because you wrote:
function series nadwat(seties y, series x, scalar bandwidth[auto],
bool leave_one_out[0], scalar trim[4])
but there is no such thing as an "auto" default value in hansl AFAIK,
Ok, something like
<hansl>
function series nadwat(seties y, series x, scalar bandwidth[0],
bool leave_one_out[0], scalar trim[4])
if bandwidth == 0
bandwidth = set_auto($nobs, x)
endif
[...]
end function
</hansl>
and also you put the default for leave_one_out to 0 whereas verbally
it
sounds as if you want to have 1. So this is confusing.
I have no strong opinion on this. Either 0 and 1 are equally tenable
default values, in my opinion.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
2:59 a.m.
Am 14.06.2018 um 08:52 schrieb Riccardo (Jack) Lucchetti:
On Thu, 14 Jun 2018, Sven Schreiber wrote:
> Am 14.06.2018 um 07:17 schrieb Riccardo (Jack) Lucchetti:
>> On Wed, 13 Jun 2018, Sven Schreiber wrote:
>>
>>> I don't see why we need to rename and deprecate the nadarwat
>>> function to achieve what you want. In the old version the user
>>> always has to give y, x and h (bandwidth). Plus in the background
>>> there is the nadarwat_trim setting (default 4).
>>>
>>> So if we extend the function as you say, then only two things need
>>> to be done internally if the function is called with only the first
>>> three args:
>>> 1) Check whether bandwidth is negative. If so, set leave_one_out to
>>> 1 and bandwidth to abs(bandwidth).
>>> 2) Check whether the nadarwat_trim value is != 4. If so, set the
>>> trim arg to that value.
>>
>> My concern is: how do you get automatic bandwidth selection and the
>> leave-one-out option together? I guess that, in your scheme, you
>> could leave the third argument empty and set the fourth one to 1,
>> but it looks sort of awkward to me. Anyway, it's a possibility.
Right, a call like nadarwat(x, y, , 1) or nadarwat(x,y, 0, 1) would be
required for that.
[BTW, before there's a misunderstanding: We're not really talking about
a hansl function, but the built-in libgretl function in C. Just using
hansl as pseudocode.]
Ok, something like
<hansl>
function series nadwat(seties y, series x, scalar bandwidth[0],
bool leave_one_out[0], scalar trim[4])
So what's the difference? I mean, how do you get your auto bandwidth and
the leave_one_out thing together. Exactly like in my proposal, no?
> and also you put the default for leave_one_out to 0 whereas
verbally
> it sounds as if you want to have 1. So this is confusing.
I have no strong opinion on this. Either 0 and 1 are equally tenable
default values, in my opinion.
OK.
cheers,
sven
6:08 a.m.
Ok, folks, I just pushed to git an extension of the nadarwat() function.
The third argument is now optional: if present, it works just like before.
If absent or zero, the bandwidth is automatically calculated as suggested
by Andreï in his message a few weeks back:
http://lists.wfu.edu/pipermail/gretl-devel/2018-May/008800.html
A fourth optional parameter is a Boolean flag to force the leave-one-out
estimator.
The following script illustrates the new syntax (not documented yet)
<hansl>
set verbose off
open mroz87.gdt --quiet
dataset sortby WA
# old_style
series o1 = nadarwat(HA, WA, 2.0) # bw = 2
series o2 = nadarwat(HA, WA, -2.0) # bw = 2, leave-one-out
series o3 = nadarwat(HA, WA, 0.05) # bw = 0.05
series o4 = nadarwat(HA, WA, -0.05) # bw = 0.05, leave-one-out
# new_style
series n2 = nadarwat(HA, WA, 2.0, 1) # bw = 2, leave-one-out
series n4 = nadarwat(HA, WA, 0.05, 1) # bw = 0.05, leave-one-out
series n_auto1 = nadarwat(HA, WA) # bw = auto
series n_auto2 = nadarwat(HA, WA, 0) # bw = auto
series n_auto3 = nadarwat(HA, WA, , 1) # bw = auto, leave-one-out
series n_auto4 = nadarwat(HA, WA, 0, 1) # bw = auto, leave-one-out
gnuplot n_auto1 n_auto3 WA --with-lines --fit=none --output=display
</hansl>
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
6:22 a.m.
Am 15.06.2018 um 12:08 schrieb Riccardo (Jack) Lucchetti:
Ok, folks, I just pushed to git an extension of the nadarwat() function.
The third argument is now optional: if present, it works just like
before. If absent or zero, the bandwidth is automatically calculated
as suggested by Andreï in his message a few weeks back:
http://lists.wfu.edu/pipermail/gretl-devel/2018-May/008800.html
A fourth optional parameter is a Boolean flag to force the
leave-one-out estimator.
Thanks Jack. You had proposed to replace the nadarwat_trim
setting by a
function argument. Any reason why you didn't do that here?
cheers,
sven
6:28 a.m.
On Fri, 15 Jun 2018, Sven Schreiber wrote:
Am 15.06.2018 um 12:08 schrieb Riccardo (Jack) Lucchetti:
>
> Ok, folks, I just pushed to git an extension of the nadarwat() function.
>
> The third argument is now optional: if present, it works just like before.
> If absent or zero, the bandwidth is automatically calculated as suggested
> by Andreï in his message a few weeks back:
> http://lists.wfu.edu/pipermail/gretl-devel/2018-May/008800.html
>
> A fourth optional parameter is a Boolean flag to force the leave-one-out
> estimator.
Thanks Jack. You had proposed to replace the nadarwat_trim setting by a
function argument. Any reason why you didn't do that here?
No particular reason other than "one thing at a time" ;)
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
6:37 a.m.
Am 15.06.2018 um 12:28 schrieb Riccardo (Jack) Lucchetti:
On Fri, 15 Jun 2018, Sven Schreiber wrote:
> Am 15.06.2018 um 12:08 schrieb Riccardo (Jack) Lucchetti:
>>
>> Ok, folks, I just pushed to git an extension of the nadarwat()
>> function.
>>
>> The third argument is now optional: if present, it works just like
>> before. If absent or zero, the bandwidth is automatically calculated
>> as suggested by Andreï in his message a few weeks back:
>> http://lists.wfu.edu/pipermail/gretl-devel/2018-May/008800.html
>>
>> A fourth optional parameter is a Boolean flag to force the
>> leave-one-out estimator.
> Thanks Jack. You had proposed to replace the nadarwat_trim setting by
> a function argument. Any reason why you didn't do that here?
No particular reason other than "one thing at a time" ;)
Right, but we don't want to have a multiple or staggered deprecations of
old behavior, do we?
-s
7:41 a.m.
On Fri, 15 Jun 2018, Sven Schreiber wrote:
Am 15.06.2018 um 12:28 schrieb Riccardo (Jack) Lucchetti:
> On Fri, 15 Jun 2018, Sven Schreiber wrote:
>
>> Am 15.06.2018 um 12:08 schrieb Riccardo (Jack) Lucchetti:
>>>
>>> Ok, folks, I just pushed to git an extension of the nadarwat() function.
>>>
>>> The third argument is now optional: if present, it works just like
>>> before. If absent or zero, the bandwidth is automatically calculated as
>>> suggested by Andreï in his message a few weeks back:
>>> http://lists.wfu.edu/pipermail/gretl-devel/2018-May/008800.html
>>>
>>> A fourth optional parameter is a Boolean flag to force the leave-one-out
>>> estimator.
>> Thanks Jack. You had proposed to replace the nadarwat_trim setting by a
>> function argument. Any reason why you didn't do that here?
>
> No particular reason other than "one thing at a time" ;)
Right, but we don't want to have a multiple or staggered deprecations of old
behavior, do we?
Sure. The one thing I don't really feel like doing (but I suppose I could
bring myself to it) is handling arguments to hansl functions with more
than three parameters. That part of the code (function eval_nargs_func in
geneval.c, if you're curious) has become really ugly through time.
Allin has had a brilliant idea for simplifying drastically that area of
the source, and possibly getting some efficiency gains too, but it'll take
some time to implement in practice (and that's definitely something I
don't feel confident about, so I don't think I can really be of any help).
That said, adding a fifth argument is defintely doable, so I'll see if I
can get it done in the next few days. Testing what we have at the moment
wouldn't be unwelcome, though (hint, hint).
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
9:32 a.m.
Am 15.06.2018 um 13:41 schrieb Riccardo (Jack) Lucchetti:
Allin has had a brilliant idea for simplifying drastically that area
of the source, and possibly getting some efficiency gains too, but
it'll take some time to implement in practice (and that's definitely
something I don't feel confident about, so I don't think I can really
be of any help).
That said, adding a fifth argument is defintely doable, so I'll see if
I can get it done in the next few days. Testing what we have at the
moment wouldn't be unwelcome, though (hint, hint).
OK.
cheers,
sven
7:04 a.m.
On Fri, 15 Jun 2018, Riccardo (Jack) Lucchetti wrote:
That said, adding a fifth argument is defintely doable, so I'll
see if I can
get it done in the next few days.
Done; it's in git now. A little test script comparing old and new syntax
follows:
<hansl>
set verbose off
open mroz87.gdt --quiet
dataset sortby WA
# old_style
series o1 = nadarwat(HA, WA, 2.0) # bw = 2
series o2 = nadarwat(HA, WA, -2.0) # bw = 2, leave-one-out
series o3 = nadarwat(HA, WA, 0.05) # bw = 0.05
series o4 = nadarwat(HA, WA, -0.05) # bw = 0.05, leave-one-out
set nadarwat_trim 2
series o5 = nadarwat(HA, WA, 2.0) # bw = 2
series o6 = nadarwat(HA, WA, -2.0) # bw = 2, leave-one-out
series o7 = nadarwat(HA, WA, 0.05) # bw = 0.05
series o8 = nadarwat(HA, WA, -0.05) # bw = 0.05, leave-one-out
# new_style
set nadarwat_trim 4
series n2 = nadarwat(HA, WA, 2.0, 1) # bw = 2, leave-one-out
series n4 = nadarwat(HA, WA, 0.05, 1) # bw = 0.05, leave-one-out
series n6 = nadarwat(HA, WA, 2.0, 1, 2) # bw = 2, leave-one-out, trim = 2
series n7 = nadarwat(HA, WA, 0.05,, 2) # bw = 0.05, trim = 2
series n_auto1 = nadarwat(HA, WA) # bw = auto
series n_auto2 = nadarwat(HA, WA, 0) # bw = auto
series n_auto3 = nadarwat(HA, WA, , 1) # bw = auto, leave-one-out
series n_auto4 = nadarwat(HA, WA, 0, 1) # bw = auto, leave-one-out
# checks
printf "check 1: %s\n", max(abs(n2 - o2)) ? "Fail" : "OK"
printf "check 2: %s\n", max(abs(n4 - o4)) ? "Fail" : "OK"
printf "check 3: %s\n", max(abs(n6 - o6)) ? "Fail" : "OK"
printf "check 4: %s\n", max(abs(n7 - o7)) ? "Fail" : "OK"
</hansl>
There are two things left to do (apart from testing):
1) update the docs
2) provide some way to have auto bandwidth choice in the GUI element
I can take care of 1) but I'm definitely out on my depth for 2). Could you
have a look, Allin?
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:28 a.m.
Am 18.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti:
There are two things left to do (apart from testing):
Great,
thanks, I will test (and maybe others, too?).
2) provide some way to have auto bandwidth choice in the GUI element
Unrelated, but looking at the current GUI output of Nadaraya-Watson
maybe it is not 100% obvious what is the dependent and independent
variable. Right after the estimation it's of course still fresh in the
user's memory, but later it might be confusing.
thanks,
sven
8:15 a.m.
Am 18.06.2018 um 13:28 schrieb Sven Schreiber:
Am 18.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti:
> There are two things left to do (apart from testing):
Great, thanks, I will test (and maybe others, too?).
What about this interpretation of inputs (current git):
<hansl>
set nadarwat_trim 4
series n2 = nadarwat(HA, WA, -2.0, 0) # bw = 2, leave-one-out or not?
series n4 = nadarwat(HA, WA, -2.0) # old style, bw = 2, leave-one-out
eval max(abs(n2-n4)) # gives 0
</hansl>
For the n2 spec there are competing user choices: old-style negative
third arg (meaning leave-one-out) and a zero fourth arg. The old style
prevails right now.
Instead I think this inconsistent input should throw an error.
thanks,
sven
8:43 a.m.
On Mon, 18 Jun 2018, Sven Schreiber wrote:
Am 18.06.2018 um 13:28 schrieb Sven Schreiber:
> Am 18.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti:
>> There are two things left to do (apart from testing):
> Great, thanks, I will test (and maybe others, too?).
What about this interpretation of inputs (current git):
<hansl>
set nadarwat_trim 4
series n2 = nadarwat(HA, WA, -2.0, 0) # bw = 2, leave-one-out or not?
series n4 = nadarwat(HA, WA, -2.0) # old style, bw = 2, leave-one-out
eval max(abs(n2-n4)) # gives 0
</hansl>
For the n2 spec there are competing user choices: old-style negative third
arg (meaning leave-one-out) and a zero fourth arg. The old style prevails
right now.
Instead I think this inconsistent input should throw an error.
The way the code works as of now is that leave-one-out can be triggered by
a negative bandwidth OR by the Boolean fourth argument. Perhaps you're
right, we should raise an error in the case of inconsistency. Other
opinions?
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:48 a.m.
On Mon, 18 Jun 2018, Riccardo (Jack) Lucchetti wrote:
On Mon, 18 Jun 2018, Sven Schreiber wrote:
> Am 18.06.2018 um 13:28 schrieb Sven Schreiber:
>> Am 18.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti:
>>> There are two things left to do (apart from testing):
>> Great, thanks, I will test (and maybe others, too?).
>
> What about this interpretation of inputs (current git):
>
> <hansl>
> set nadarwat_trim 4
> series n2 = nadarwat(HA, WA, -2.0, 0) # bw = 2, leave-one-out or not?
> series n4 = nadarwat(HA, WA, -2.0) # old style, bw = 2, leave-one-out
>
> eval max(abs(n2-n4)) # gives 0
> </hansl>
>
> For the n2 spec there are competing user choices: old-style negative third
> arg (meaning leave-one-out) and a zero fourth arg. The old style prevails
> right now.
> Instead I think this inconsistent input should throw an error.
The way the code works as of now is that leave-one-out can be triggered by a
negative bandwidth OR by the Boolean fourth argument. Perhaps you're right,
we should raise an error in the case of inconsistency. Other opinions?
I think under the new scheme a negative bandwidth should not be
accepted when more than three arguments are given. For backward
compatibility it could still be accepted, with its original meaning,
whhen just three arguments are given.
Allin
11:03 a.m.
On Mon, 18 Jun 2018, Allin Cottrell wrote:
On Mon, 18 Jun 2018, Riccardo (Jack) Lucchetti wrote:
> On Mon, 18 Jun 2018, Sven Schreiber wrote:
>
>> Am 18.06.2018 um 13:28 schrieb Sven Schreiber:
>>> Am 18.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti:
>>>> There are two things left to do (apart from testing):
>>> Great, thanks, I will test (and maybe others, too?).
>>
>> What about this interpretation of inputs (current git):
>>
>> <hansl>
>> set nadarwat_trim 4
>> series n2 = nadarwat(HA, WA, -2.0, 0) # bw = 2, leave-one-out or not?
>> series n4 = nadarwat(HA, WA, -2.0) # old style, bw = 2, leave-one-out
>>
>> eval max(abs(n2-n4)) # gives 0
>> </hansl>
>>
>> For the n2 spec there are competing user choices: old-style negative third
>> arg (meaning leave-one-out) and a zero fourth arg. The old style prevails
>> right now.
>> Instead I think this inconsistent input should throw an error.
>
> The way the code works as of now is that leave-one-out can be triggered by
> a negative bandwidth OR by the Boolean fourth argument. Perhaps you're
> right, we should raise an error in the case of inconsistency. Other
> opinions?
I think under the new scheme a negative bandwidth should not be accepted when
more than three arguments are given. For backward compatibility it could
still be accepted, with its original meaning, whhen just three arguments are
given.
Makes sense. Now in git.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
9:42 a.m.
Am 18.06.2018 um 17:03 schrieb Riccardo (Jack) Lucchetti:
On Mon, 18 Jun 2018, Allin Cottrell wrote:
> On Mon, 18 Jun 2018, Riccardo (Jack) Lucchetti wrote:
>
>> On Mon, 18 Jun 2018, Sven Schreiber wrote:
>>
>>> Instead I think this inconsistent input should throw an error.
>>
>> The way the code works as of now is that leave-one-out can be
>> triggered by a negative bandwidth OR by the Boolean fourth argument.
>> Perhaps you're right, we should raise an error in the case of
>> inconsistency. Other opinions?
>
> I think under the new scheme a negative bandwidth should not be
> accepted when more than three arguments are given. For backward
> compatibility it could still be accepted, with its original meaning,
> whhen just three arguments are given.
Makes sense. Now in git.
OK. Then a related question is how to deal with the nadarwat_trim
setting when the fifth (trimming) argument is given. My suggestion would
be to print a deprecation warning in the case where (i) the
nadarwat_trim setting has been user-adjusted (to a non-default value)
and (ii) a different value is given as the fifth arg.
But I admit that maybe it's too much of a fringe issue.
thanks,
sven
2343
days inactive
2536
days old
31 comments
4 participants
participants (4)
-
Allin Cottrell -
Andreï V. Kostyrka -
Riccardo (Jack) Lucchetti -
Sven Schreiber