2:35 p.m.
Hi,
I have just revisited a Midas script of mine, and when I saw something like:
midasreg dep const ownlag; mds(dIP, 0,4, 1, theta)
I was scratching my head and asked myself, OK what on earth was that
specification again? Gretl's Midas doc of course reveals that the "1" in
there means "normalized exponential Almon", but this is really not user
friendly I dare to say.
I suggest that instead of an integer code there should be string aliases
like "nexpAlmon" (and e.g. "u" for unrestricted, "Almon",
"betalast0",
"beta").
Would that --or something similar-- be doable?
Thanks,
sven
4:19 p.m.
Am 12.09.2017 um 16:35 schrieb Sven Schreiber:
midasreg dep const ownlag; mds(dIP, 0,4, 1, theta)
P.S.: And what about a default value for theta (vector of initial
values)? So far I'm always setting the initial values to zero, for lack
of better ideas -- perhaps there are more reasonable values, but it
would be useful if there was a default. Then we could write instead:
midasreg dep const ownlag; mds(dIP, 0,4, "nexpAlmon")
thanks,
sven
4:31 p.m.
Am 13.09.2017 um 18:19 schrieb Sven Schreiber:
P.S.: And what about a default value for theta (vector of initial
values)? So far I'm always setting the initial values to zero, for lack
of better ideas -- perhaps there are more reasonable values, but it
would be useful if there was a default. Then we could write instead:
midasreg dep const ownlag; mds(dIP, 0,4, "nexpAlmon")
Sorry, to avoid confusion: According to the doc there is currently
already the possibility to write "null" for the specification types 1-3.
(For spec type 0 it's already optional.) What I mean is if this explicit
"null" could also be made optional.
thanks,
sven
7:44 p.m.
I don't want to open too many Midas threads, so I'm kind of hijacking my
own: Here's another issue, but this time it looks like a bug to me.
From what I understand, the polynomial specification beta-zero-last-lag
(type code 2) should give a coefficient weight of zero for the last lag,
hence its name. (The only exception should be the case when the two
hyperparameters are both one, theta = {1, 1}.)
However, consider these examples:
<hansl>
eval mweights(10, {0,0}, 2)' # gives 0.5 at the end
eval mweights(10, {0.5, 0.5}, 2)' # dito
eval mweights(10, {1.1, 1}, 2)' # gives 0.1194
eval mweights(10, {2, 1}, 2)' # gives 0.2
</hansl>
Am I missing something?
If it's really a bug, does it only affect the mweights() function or
does it spill over into actual midasreg results?
This is with an August snapshot.
Thanks,
sven
9:01 p.m.
On Wed, 13 Sep 2017, Sven Schreiber wrote:
I don't want to open too many Midas threads, so I'm kind of
hijacking my own: Here's another issue, but this time it looks
like a bug to me.
From what I understand, the polynomial specification
beta-zero-last-lag (type code 2) should give a coefficient weight
of zero for the last lag, hence its name. (The only exception
should be the case when the two hyperparameters are both one,
theta = {1, 1}.)
However, consider these examples:
<hansl>
eval mweights(10, {0,0}, 2)' # gives 0.5 at the end
eval mweights(10, {0.5, 0.5}, 2)' # dito
eval mweights(10, {1.1, 1}, 2)' # gives 0.1194
eval mweights(10, {2, 1}, 2)' # gives 0.2
</hansl>
Am I missing something?
If it's really a bug, does it only affect the mweights() function or does it
spill over into actual midasreg results?
Hmm. I've checked, and our "beta-with-zero-last-lag" weights agree
with those produced by Ghysels' MIDAS Matlab Toolbox and also midasr
(modulo the fact that midasr for some reason makes the weights sum
to the number of hyperparameters rather than to 1.0, as with Matlab
and gretl).
So our calculations seem to be right, but apparently the doc is not
entirely accurate -- an inaccuracy that is inherited from the Matlab
doc. It seems that, besides the {1, 1} case, "pathological"
hyperparameter vectors can give a non-zero last lag. Looking at the
weights implied by your 4 cases above, they are somewhat strange.
{0, 0}: All weights are machine-zero apart from the first and last.
{0.5, 0.5}: All pretty much ~= 0 apart from first and last.
{1.1, 1}: weights increase progressively at longer lags.
{2, 1}: ditto.
My impression is that "sane" values of theta have theta_1 around 1.0
and theta_2 > theta_1 -- and that such vectors will respect the
"zero last lag" condition. Admittedly, this is not explicitly stated
anywhere (that I'm aware of).
Allin
9:21 p.m.
Am 13.09.2017 um 23:01 schrieb Allin Cottrell:
My impression is that "sane" values of theta have theta_1
around 1.0 and
theta_2 > theta_1 -- and that such vectors will respect the "zero last
lag" condition. Admittedly, this is not explicitly stated anywhere (that
I'm aware of).
Right, but then the estimation can steer the hyperparameters into the
"unsane" region, can't it?
I had also thought about whether there is a condition like th1 < th2 and
looked for it in Ghysels' guide, but couldn't find anything.
(And BTW, what you report on the weighting/summing in R's midasr sounds
weird.)
thanks,
sven
9:38 p.m.
On Wed, 13 Sep 2017, Sven Schreiber wrote:
Am 13.09.2017 um 23:01 schrieb Allin Cottrell:
> My impression is that "sane" values of theta have theta_1 around 1.0 and
> theta_2 > theta_1 -- and that such vectors will respect the "zero last
lag"
> condition. Admittedly, this is not explicitly stated anywhere (that I'm
> aware of).
Right, but then the estimation can steer the hyperparameters into the
"unsane" region, can't it?
In principle, I suppose, but I haven't seen it happen.
I had also thought about whether there is a condition like th1 <
th2 and
looked for it in Ghysels' guide, but couldn't find anything.
Nor could I.
(And BTW, what you report on the weighting/summing in R's midasr
sounds
weird.)
Yep, it seems strange to me too. But it's definitely the case: the
weights returned by midasr's nbeta() function -- and its analogs for
the other parameterizations -- sum to the length of the vector of
hyperparameters. So typically they are twice the size of those given
by Matlab and gretl.
Allin
7:50 a.m.
Am 13.09.2017 um 23:38 schrieb Allin Cottrell:
On Wed, 13 Sep 2017, Sven Schreiber wrote:
> Am 13.09.2017 um 23:01 schrieb Allin Cottrell:
>
>> My impression is that "sane" values of theta have theta_1 around 1.0
>> and theta_2 > theta_1 -- and that such vectors will respect the "zero
>> last lag" condition. Admittedly, this is not explicitly stated
>> anywhere (that I'm aware of).
>
> Right, but then the estimation can steer the hyperparameters into the
> "unsane" region, can't it?
In principle, I suppose, but I haven't seen it happen.
Well I have a case here where I set the initial values to {0.5, 0}, and
that's where the routine converges. The results have various obvious and
visible problems, but I think it constitutes a proof of concept.
So I'm wondering whether a condition like th2 > th1 should be imposed at
least on the starting values. (It's more difficult to impose it overall
I guess.) But of course it would be nice to have some literature on that.
And while I experimented with those Midas weights, another question came
up: Is $model.seplist after midasreg properly implemented? Consider this
example:
<hansl>
open gdp_midas.gdt
list mi = ld_indpro_m?
list mp = ld_payems_m?
midasreg ld_qgdp const ld_qgdp(-1) ; \
mds(mi, 0,2, 2, null) mds(mp, 1,5, 2, null)
check = $model.seplist
list check print # I get: ld_qgdp payems_m1
</hansl>
Note that the members of 'check' have ID numbers 2 and 5 in the dataset
-- and that the first set of Midas coefficients is in positions 2-4, the
second set is in positions 5-7. So I'm guessing that this provides
information where the Midas coefficients are located in the $coeff
vector, but somehow this is erroneously (?) put into a list container?
Also, $model does not seem to contain information about the hi-freq
regressors: $model.xlist gives me the same as $model.lfxlist.
thanks,
sven
9:20 a.m.
Am 13.09.2017 um 23:38 schrieb Allin Cottrell:
On Wed, 13 Sep 2017, Sven Schreiber wrote:
> Am 13.09.2017 um 23:01 schrieb Allin Cottrell:
>> My impression is that "sane" values of theta have theta_1 around 1.0
>> and theta_2 > theta_1 -- and that such vectors will respect the "zero
>> last lag" condition.
Hm, yes and no. I have manually coded the
equations 2.10/2.12 in
Ghysels' user guide (version 8), that is the zero-last-lag beta case,
and _sometimes_ (but not always) get different results from gretl's
mweights() function with type==2. For example:
<hansl>
function matrix weightcrosscheck(int N, scalar th1, scalar th2)
matrix w = -1 * ones(N,1)
denom = 0
loop i=1..N --quiet
x = (i-1) / (N-1)
w[i] = x^(th1 - 1) * (1-x)^(th2 - 1)
denom += w[i]
endloop
w = w ./ denom
return w
end function
N = 9
## mismatch:
th1 = 1.01
th2 = 1.02
eval weightcrosscheck(N, th1, th2) # last weight: 0.0000
eval mweights(N, {th1, th2}, 2) # last weight: 0.0608
## a matching case (but not th1 < th2):
th1 = 2
th2 = 1
eval weightcrosscheck(N, th1, th2) # last weight: 0.22
eval mweights(N, {th1, th2}, 2) # last weight: 0.22
</hansl>
> (And BTW, what you report on the weighting/summing in R's
midasr
> sounds weird.)
Yep, it seems strange to me too. But it's definitely the case: the
weights returned by midasr's nbeta() function -- and its analogs for the
other parameterizations -- sum to the length of the vector of
hyperparameters. So typically they are twice the size of those given by
Matlab and gretl.
On second thought this might just be an arbitrary identification choice.
If in gretl Sum == 1 and in midasr SumR == 2 then this just means that
the slope hyperparameter in midasr would be half that of gretl, but the
"gross" polynomial coefficients could still be äquivalent.
cheers,
sven
4:56 p.m.
On Wed, 13 Sep 2017, Sven Schreiber wrote:
Am 13.09.2017 um 18:19 schrieb Sven Schreiber:
> P.S.: And what about a default value for theta (vector of initial values)?
> So far I'm always setting the initial values to zero, for lack of better
> ideas -- perhaps there are more reasonable values, but it would be useful
> if there was a default. Then we could write instead:
>
> midasreg dep const ownlag; mds(dIP, 0,4, "nexpAlmon")
Sorry, to avoid confusion: According to the doc there is currently already
the possibility to write "null" for the specification types 1-3. (For spec
type 0 it's already optional.) What I mean is if this explicit "null" could
also be made optional.
That would be a bit awkward to arrange. However, in git we now support
use of names in place of numeric codes for the parameterizations as
follows:
0: "umidas"
1: "nealmon"
2: "beta0"
3: "betan"
4: "almonp"
Allin
5:26 p.m.
On Thu, 14 Sep 2017, Sven Schreiber wrote:
Am 13.09.2017 um 23:38 schrieb Allin Cottrell:
> On Wed, 13 Sep 2017, Sven Schreiber wrote:
>
>> Am 13.09.2017 um 23:01 schrieb Allin Cottrell:
>>> My impression is that "sane" values of theta have theta_1 around
1.0
>>> and theta_2 > theta_1 -- and that such vectors will respect the "zero
>>> last lag" condition.
Hm, yes and no. I have manually coded the equations 2.10/2.12 in
Ghysels' user guide (version 8), that is the zero-last-lag beta case,
and _sometimes_ (but not always) get different results from gretl's
mweights() function with type==2. For example:
<hansl>
function matrix weightcrosscheck(int N, scalar th1, scalar th2)
matrix w = -1 * ones(N,1)
denom = 0
loop i=1..N --quiet
x = (i-1) / (N-1)
w[i] = x^(th1 - 1) * (1-x)^(th2 - 1)
denom += w[i]
endloop
w = w ./ denom
return w
end function
N = 9
## mismatch:
th1 = 1.01
th2 = 1.02
eval weightcrosscheck(N, th1, th2) # last weight: 0.0000
eval mweights(N, {th1, th2}, 2) # last weight: 0.0608
## a matching case (but not th1 < th2):
th1 = 2
th2 = 1
eval weightcrosscheck(N, th1, th2) # last weight: 0.22
eval mweights(N, {th1, th2}, 2) # last weight: 0.22
</hansl>
Gretl's mweights() results agree with those from Ghysels' Matlab code,
which differ from your weightcrosscheck. Here's the function he uses:
<matlab>
function [be_values]=b_weights(ind,k1,k2)
%Beta weights function.
if length(ind)>1
u=(ind(1)-ind)/(ind(1)-ind(end));
u(1)=u(1)+eps;
u(end)=u(end)-eps;
end
if isscalar(ind)
u=linspace(eps,1-eps,ind)';
end
be_values=u.^(k1-1).*(1-u).^(k2-1);
be_values=be_values/sum(be_values);
end
</matlab>
Octave gives the following for b_weights(9, 1.01, 1.02):
0.087114
0.122020
0.122491
0.122541
0.122347
0.121916
0.121152
0.119669
0.060751
Allin
5:55 p.m.
On Thu, 14 Sep 2017, Sven Schreiber wrote:
And while I experimented with those Midas weights, another question
came up:
Is $model.seplist after midasreg properly implemented? Consider this example:
<hansl>
open gdp_midas.gdt
list mi = ld_indpro_m?
list mp = ld_payems_m?
midasreg ld_qgdp const ld_qgdp(-1) ; \
mds(mi, 0,2, 2, null) mds(mp, 1,5, 2, null)
check = $model.seplist
list check print # I get: ld_qgdp payems_m1
</hansl>
Note that the members of 'check' have ID numbers 2 and 5 in the
dataset -- and that the first set of Midas coefficients is in
positions 2-4, the second set is in positions 5-7. So I'm guessing
that this provides information where the Midas coefficients are
located in the $coeff vector,
Yes, that's right.
but somehow this is erroneously (?) put into a list container?
Well, the list container is convenient for internal use. But since
it's not a "list" in the usual gretl sense it should probably be
omitted from the $model bundle.
Also, $model does not seem to contain information about the hi-freq
regressors: $model.xlist gives me the same as $model.lfxlist.
The information on the hi-freq terms is stored on the model struct in
the form of an array of bundles. This array wasn't getting saved into
the $model bundle but that's now corrected in git.
midasreg <whatever>
bundles B = $model.midas_info
loop i=1..nelem(B) -q
eval B[i]
endloop
Allin
7:36 p.m.
Am 14.09.2017 um 19:55 schrieb Allin Cottrell:
On Thu, 14 Sep 2017, Sven Schreiber wrote:
> Also, $model does not seem to contain information about the
hi-freq
> regressors: $model.xlist gives me the same as $model.lfxlist.
The information on the hi-freq terms is stored on the model struct in
the form of an array of bundles. This array wasn't getting saved into
the $model bundle but that's now corrected in git.
Ah, very good!
thanks,
sven
7:43 p.m.
Am 14.09.2017 um 19:26 schrieb Allin Cottrell:
On Thu, 14 Sep 2017, Sven Schreiber wrote:
Gretl's mweights() results agree with those from Ghysels'
Matlab code,
which differ from your weightcrosscheck. Here's the function he uses:
Octave gives the following for b_weights(9, 1.01, 1.02):
...
0.060751
Yes, I was kind of expecting that my function is the odd one out.
However, the point is that we're talking about something where
supposedly the "last lag is zero" by construction, and apparently only
my naive implementation actually satisfies that. (Even for what we have
declared a "sane" case, theta1 \approx 1 and theta1 < theta2.)
So this leaves me even more puzzled than before.
thanks,
sven
7:44 p.m.
Am 14.09.2017 um 18:56 schrieb Allin Cottrell:
On Wed, 13 Sep 2017, Sven Schreiber wrote:
> 1-3. (For spec type 0 it's already optional.) What I mean is
if this
> explicit "null" could also be made optional.
That would be a bit awkward to arrange. However, in git we now support
use of names in place of numeric codes for the parameterizations as
follows:
0: "umidas"
1: "nealmon"
2: "beta0"
3: "betan"
4: "almonp"
thanks, I think this is very good for readability!
-sven
8:09 p.m.
On Thu, 14 Sep 2017, Allin Cottrell wrote:
On Wed, 13 Sep 2017, Sven Schreiber wrote:
> Am 13.09.2017 um 18:19 schrieb Sven Schreiber:
>
>> P.S.: And what about a default value for theta (vector of initial values)?
>> So far I'm always setting the initial values to zero, for lack of better
>> ideas -- perhaps there are more reasonable values, but it would be useful
>> if there was a default. Then we could write instead:
>>
>> midasreg dep const ownlag; mds(dIP, 0,4, "nexpAlmon")
>
> Sorry, to avoid confusion: According to the doc there is currently already
> the possibility to write "null" for the specification types 1-3. (For spec
> type 0 it's already optional.) What I mean is if this explicit "null"
could
> also be made optional.
That would be a bit awkward to arrange. [...]
OK, not really all that awkward. In git we now support dropping the
"theta" (hyperparameter initialization) argument to mds() and mdsl()
altogether, meaning that the initialization is automatic (as if you
gave "null" for the argument).
I'll update the doc on this point, and others, before too long.
Allin
9:12 p.m.
Am 14.09.2017 um 19:26 schrieb Allin Cottrell:
if isscalar(ind)
u=linspace(eps,1-eps,ind)';
end
be_values=u.^(k1-1).*(1-u).^(k2-1);
OK, now let's check the influence of this 'eps' thingy which according
to Ghysels is there for numerical stability AFAIU. (Here k2 == 1.02 so
k2-1 == 0.02, and I'm focusing on the final element in the u vector.)
<octave-session-part>
> u = 1 - eps
u = 1.00000
> (1-u) .^ 0.02
ans = 0.48633
> (1-1) .^ 0.02
ans = 0
>
</octave-session-part>
Thus if we put in a plain and simple 1, then we get the "blackboard
math" solution 0 for the last lag. If instead we use a fancy/fuzzy
1-eps, this turns into 0.49!
So I'm wondering whether the eps cure isn't worse than the original illness?
cheers,
sven
9:50 p.m.
On Thu, 14 Sep 2017, Sven Schreiber wrote:
Am 14.09.2017 um 19:26 schrieb Allin Cottrell:
> if isscalar(ind)
> u=linspace(eps,1-eps,ind)';
> end
>
> be_values=u.^(k1-1).*(1-u).^(k2-1);
OK, now let's check the influence of this 'eps' thingy which according
to Ghysels is there for numerical stability AFAIU. (Here k2 == 1.02 so
k2-1 == 0.02, and I'm focusing on the final element in the u vector.)
<octave-session-part>
>> u = 1 - eps
u = 1.00000
>> (1-u) .^ 0.02
ans = 0.48633
>> (1-1) .^ 0.02
ans = 0
>>
</octave-session-part>
Thus if we put in a plain and simple 1, then we get the "blackboard
math" solution 0 for the last lag. If instead we use a fancy/fuzzy
1-eps, this turns into 0.49!
So I'm wondering whether the eps cure isn't worse than the original illness?
I agree with you this far: the role of "eps" in Ghysels' code is not
transparent to me; I'm not sure what the "numerical stability" that
it's said to promote really means here.
However, in relation to your example it appears that maybe it's
designed to prevent abrupt changes in weight at the ends of the lag
range. As you compute the weights they jump from and to zero at the
end-points; as Ghysels computes them the profile is smoother.
Actually meeting the advertised condition of "zero last lag" -- on
Ghysels' method -- requires that the coefficients are progressively
shrinking as we near the maximum lag. And that is achieved only if
theta[2] is substantially greater than theta[1]. (For example,
theta[1] = 1, theta[2] >= 2.)
Allin
7:27 a.m.
Am 14.09.2017 um 23:50 schrieb Allin Cottrell:
However, in relation to your example it appears that maybe it's
designed
to prevent abrupt changes in weight at the ends of the lag range. As you
compute the weights they jump from and to zero at the end-points; as
Ghysels computes them the profile is smoother.
Well, there will _always_ be abrupt jumps to zero, namely at lag N+1. If
anybody wants to have a smooth function, no problem with me, but then
IMHO they shouldn't call this "last lag zero" if that's not what it
does.
We have inferred (or guessed) some of the implicitly sane restrictions
on the thetas: e.g. not (th2 == 1 && th1 != 1), because 0^0 == 1, as
well as (th2 > th1 || th2 == 1). Then I think this should be imposed at
least on the starting values (easy) and perhaps also during the
optimization (maybe not so easy).
Given the advertised spec of "last lag zero", frankly the original
implementation just looks buggy to me right now, and by implication
gretl's imitation as well. At the very least the user could be warned
that "last lag" sometimes refers to N but sometimes might mean just N+1.
But again, having zero at N+1 is of course trivially true.
Actually meeting the advertised condition of "zero last
lag" -- on
Ghysels' method -- requires that the coefficients are progressively
shrinking as we near the maximum lag. And that is achieved only if
theta[2] is substantially greater than theta[1]. (For example, theta[1]
= 1, theta[2] >= 2.)
Any ideas what the R package or Eviews (>= 9.5) do? The Eviews
documentation was not so illuminating in that detail, but I haven't
checked the programming reference yet, maybe there's something there.
thanks,
sven
9:01 a.m.
Am 14.09.2017 um 21:44 schrieb Sven Schreiber:
Am 14.09.2017 um 18:56 schrieb Allin Cottrell:
> follows:
>
> 0: "umidas"
> 1: "nealmon"
> 2: "beta0"
> 3: "betan"
> 4: "almonp"
thanks, I think this is very good for readability!
Not sure if gretl should copy Ghysels' package there, but I guess it's
now or never. In his user guide I find the strings (for opt.for.poly):
'umidas'
'beta'
'betann' (non-zero)
'almon'
Not sure which of the Almons 'almon' refers to, though.
cheers,
sven
12:19 p.m.
On Fri, 15 Sep 2017, Sven Schreiber wrote:
Am 14.09.2017 um 23:50 schrieb Allin Cottrell:
> However, in relation to your example it appears that maybe it's designed to
> prevent abrupt changes in weight at the ends of the lag range. As you
> compute the weights they jump from and to zero at the end-points; as
> Ghysels computes them the profile is smoother.
Well, there will _always_ be abrupt jumps to zero, namely at lag N+1.
The jump will not be very abrupt if the prior coefficients are
declining, which seems to be taken for granted.
[...]
Any ideas what the R package or Eviews (>= 9.5) do?
No idea what Eviews does, though I suspect it will mimic Ghysels. R's
nbeta() produces the same results as Ghysels and gretl.
<hansl>
eval mweights(9, {1.01,1.02}, 2)
foreign language=R
library(midasr)
b <- nbeta(p=c(2, 1.01, 1.02), d=9)
b/2.0
end foreign
</hansl>
Allin
1:21 p.m.
Am 15.09.2017 um 14:19 schrieb Allin Cottrell:
On Fri, 15 Sep 2017, Sven Schreiber wrote:
> Am 14.09.2017 um 23:50 schrieb Allin Cottrell:
>
>> However, in relation to your example it appears that maybe it's
>> designed to prevent abrupt changes in weight at the ends of the lag
>> range. As you compute the weights they jump from and to zero at the
>> end-points; as Ghysels computes them the profile is smoother.
>
> Well, there will _always_ be abrupt jumps to zero, namely at lag N+1.
The jump will not be very abrupt if the prior coefficients are
declining, which seems to be taken for granted.
Well, except in other cases that we have looked at, like th1==2 and
th==1 (which doesn't have to do with the eps thing, of course), where
both implementations jump from (the max weight) 0.1 at N to 0 at N+1.
My personal takeaway is that this beta0/betann is unreliable and/or
misleading by construction. I'd rather go with the other beta or perhaps
with the Almons (not to mention umidas).
Eventually I will try if I can find out more for Eviews.
thanks,
sven
3:57 p.m.
On Fri, 15 Sep 2017, Sven Schreiber wrote:
Am 15.09.2017 um 14:19 schrieb Allin Cottrell:
> On Fri, 15 Sep 2017, Sven Schreiber wrote:
>
>> Am 14.09.2017 um 23:50 schrieb Allin Cottrell:
>>
>>> However, in relation to your example it appears that maybe it's designed
>>> to prevent abrupt changes in weight at the ends of the lag range. As you
>>> compute the weights they jump from and to zero at the end-points; as
>>> Ghysels computes them the profile is smoother.
>>
>> Well, there will _always_ be abrupt jumps to zero, namely at lag N+1.
>
> The jump will not be very abrupt if the prior coefficients are declining,
> which seems to be taken for granted.
Well, except in other cases that we have looked at, like th1==2 and th==1
(which doesn't have to do with the eps thing, of course), where both
implementations jump from (the max weight) 0.1 at N to 0 at N+1.
My personal takeaway is that this beta0/betann is unreliable and/or
misleading by construction. I'd rather go with the other beta or perhaps with
the Almons (not to mention umidas).
One more comment from me: by "seems to be taken for granted" above I
meant that it seems to be taken for granted that suitable beta
hyperparameters for MIDAS are going to exhibit theta2 sufficiently
large in relation to theta1 as to ensure that the weights decline at
higher lags (possibly after an initial hump). And in that case the
"zero last lag" (two-param) version really does give a zero last lag.
If, on the contrary, the params are such that the weights are flat or
even increasing at higher lags, then "zero last lag" does not get
imposed by Ghysels' algorithm, but I guess that means there's
misspecification: not enough lags have been included.
Allin
2:24 p.m.
Am 15.09.2017 um 17:57 schrieb Allin Cottrell:
One more comment from me: by "seems to be taken for
granted" above I
meant that it seems to be taken for granted that suitable beta
hyperparameters for MIDAS are going to exhibit theta2 sufficiently large
in relation to theta1 as to ensure that the weights decline at higher
lags (possibly after an initial hump). And in that case the
"zero last lag" (two-param) version really does give a zero last lag.
Right. The question is whether it really is appropriate to skip any
checks or explicit restrictions. I'll try to keep my eyes open in
real-world cases.
If, on the contrary, the params are such that the weights are flat or
even increasing at higher lags, then "zero last lag" does not get
imposed by Ghysels' algorithm, but I guess that means there's
misspecification: not enough lags have been included.
Ok, interesting perspective on the issue.
thanks,
sven
2:41 p.m.
Am 14.09.2017 um 19:55 schrieb Allin Cottrell:
On Thu, 14 Sep 2017, Sven Schreiber wrote:
The information on the hi-freq terms is stored on the model struct in
the form of an array of bundles. This array wasn't getting saved into
the $model bundle but that's now corrected in git.
I've just checked with the snapshot from Sep-15. (BTW, I had to use the
32bit version, because the latest 64bit snapshot was still from Sep-13.)
Two or three questions remain about it, using the following example.
<hansl>
open gdp_midas
list mi = ld_indpro_m?
list mp = ld_payems_m?
midasreg ld_qgdp const ld_qgdp(-1) ; \
mds(mi, 0,2, 2, null) mds(mp, 1,5, 2, null)
bundles B = $model.midas_info
bundle b = B[1]
print b # gives 'nparm = 2', 'mname = theta___1' etc.
eval typestr(typeof(b.lname)) # gives 'list'
eval typestr(typeof(b.mname)) # gives 'null'
bundle bb = B[2]
print bb
</hansl>
First, what about the 'nparm' bundle element? Here it says 2, but there
are actually three parameters in each of these midas groups: HF_slope1,
Beta1, Beta2. Is this a bug or am I supposed to infer the number of
parameters also from the 'type' information?
Second, the 'mname' member contains something unusable apparently as
seen by the typeof() output, is this intended to be there?
Third, there is a 'prelag' bundle member -- do I guess correctly that
prelag==0 refers to a mds()-style specification and prelag==1 is for a
mdsl() thing?
Fourth, a comment/request rather than a question: Would it be possible
to save in each bundle in $model.midas_info directly the start/end
indices of the associated coefficients in the $coeff vector? I know I
can do the accounting myself, but that requires an extra loop AFAICS.
Thanks,
sven
6:30 p.m.
Am 13.09.2017 um 21:44 schrieb Sven Schreiber:
I don't want to open too many Midas threads, so I'm kind of
hijacking my
own: Here's another issue, but this time it looks like a bug to me.
And yet another Midas thing, is it intended that the lag numbers in the
mds() terms cannot be given as integer variables? See this example:
<hansl>
open gdp_midas
list mi = ld_indpro_m?
midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, 0,2, 0) # works
# The same with variables
start = 0
stop = 2
midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, start,stop, 0) # fails
</hansl>
Maybe we talked about something like this in Athens, I don't remember
the details.
Thanks,
sven
8:25 p.m.
On Sun, 17 Sep 2017, Sven Schreiber wrote:
Am 14.09.2017 um 19:55 schrieb Allin Cottrell:
> On Thu, 14 Sep 2017, Sven Schreiber wrote:
> The information on the hi-freq terms is stored on the model struct in the
> form of an array of bundles. This array wasn't getting saved into the
> $model bundle but that's now corrected in git.
I've just checked with the snapshot from Sep-15. (BTW, I had to use the
32bit version, because the latest 64bit snapshot was still from Sep-13.)
Glad you spotted that. There was a problem with the packaging
mechanism (now fixed).
Two or three questions remain about it, using the following example.
<hansl>
open gdp_midas
list mi = ld_indpro_m?
list mp = ld_payems_m?
midasreg ld_qgdp const ld_qgdp(-1) ; \
mds(mi, 0,2, 2, null) mds(mp, 1,5, 2, null)
bundles B = $model.midas_info
bundle b = B[1]
print b # gives 'nparm = 2', 'mname = theta___1' etc.
eval typestr(typeof(b.lname)) # gives 'list'
eval typestr(typeof(b.mname)) # gives 'null'
bundle bb = B[2]
print bb
</hansl>
First, what about the 'nparm' bundle element? Here it says 2, but
there are actually three parameters in each of these midas groups:
HF_slope1, Beta1, Beta2. Is this a bug or am I supposed to infer
the number of parameters also from the 'type' information?
The latter. If the parameterization is such as to require an
"HF_slope" term, that's taken for granted. (But this information was
really intended for internal use, where we know what's being
assumed.)
Second, the 'mname' member contains something unusable
apparently
as seen by the typeof() output, is this intended to be there?
"mname" may record the name of a matrix that was passed in by the
user, or it may refer to an internally constructed temporary matrix
that (as in this case) has passed out of existence by the time the
bundle is evaluated. I guess recording mname in the second case may
be redundant.
Third, there is a 'prelag' bundle member -- do I guess
correctly
that prelag==0 refers to a mds()-style specification and prelag==1
is for a mdsl() thing?
You do guess correctly.
Fourth, a comment/request rather than a question: Would it be
possible to save in each bundle in $model.midas_info directly the
start/end indices of the associated coefficients in the $coeff
vector? I know I can do the accounting myself, but that requires
an extra loop AFAICS.
Yes, we could do that, if this sort of information is going to be
useful to script writers.
Allin
8:46 p.m.
On Sun, 17 Sep 2017, Sven Schreiber wrote:
Am 13.09.2017 um 21:44 schrieb Sven Schreiber:
> I don't want to open too many Midas threads, so I'm kind of hijacking my
> own: Here's another issue, but this time it looks like a bug to me.
And yet another Midas thing, is it intended that the lag numbers
in the mds() terms cannot be given as integer variables? See this
example:
<hansl>
open gdp_midas
list mi = ld_indpro_m?
midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, 0,2, 0) # works
# The same with variables
start = 0
stop = 2
midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, start,stop, 0) # fails
</hansl>
Intended? Well, it's just not implemented (yet). I kept some things
simple in a first-pass implementation.
There's a certain amount of redundancy here: do we actually need
both mds() and mdsl() anyway? The hflags() function lets you
specify minlag and maxlag any way you wish.
Allin
9:11 p.m.
Am 17.09.2017 um 22:46 schrieb Allin Cottrell:
There's a certain amount of redundancy here: do we actually need
both
mds() and mdsl() anyway? The hflags() function lets you specify minlag
and maxlag any way you wish.
That's right. I'd even say that the extra letters 'mds' might already be
redundant, given that the Midas context is already given by the
'midasreg' command.
So instead of this:
list gip = ld_indpro_m?
midasreg ld_qgdp const ld_qgdp(-1) ; mds(gip, 0,2, "umidas")
one might have:
midasreg ld_qgdp const ld_qgdp(-1) ; gip(0,2, "umidas")
or perhaps:
list giplags = hflags(gip, 0,5)
midasreg ld_qgdp const ld_qgdp(-1) ; giplags("beta0")
Maybe the argument ordering may have to be reshuffled.
cheers,
sven
9:22 p.m.
On Sun, 17 Sep 2017, Allin Cottrell wrote:
On Sun, 17 Sep 2017, Sven Schreiber wrote:
> Am 13.09.2017 um 21:44 schrieb Sven Schreiber:
>> I don't want to open too many Midas threads, so I'm kind of hijacking my
>> own: Here's another issue, but this time it looks like a bug to me.
>
> And yet another Midas thing, is it intended that the lag numbers in the
> mds() terms cannot be given as integer variables? See this example:
>
> <hansl>
> open gdp_midas
> list mi = ld_indpro_m?
>
> midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, 0,2, 0) # works
>
> # The same with variables
> start = 0
> stop = 2
> midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, start,stop, 0) # fails
> </hansl>
Intended? Well, it's just not implemented (yet).
It's implemented now, in git. The min and max lags for a high
frequency series in the context of a mds() term under "midasreg" may
be given as either integers in numerical form or the names of scalar
variables with integer values (though not as arbitrary expressions
that would evaluate to integers).
Allin
9:07 a.m.
Am 18.09.2017 um 23:22 schrieb Allin Cottrell:
On Sun, 17 Sep 2017, Allin Cottrell wrote:
> On Sun, 17 Sep 2017, Sven Schreiber wrote:
>> midasreg ld_qgdp const ld_qgdp(-1) ; mds(mi, start,stop, 0) #
fails
>> </hansl>
>
> Intended? Well, it's just not implemented (yet).
It's implemented now, in git.
Ah, I was already settled on moving over to the mdsl() variant
completely. But nice to have that, of course, thanks.
-sven
1:42 p.m.
Am 14.09.2017 um 22:09 schrieb Allin Cottrell:
On Thu, 14 Sep 2017, Allin Cottrell wrote:
> On Wed, 13 Sep 2017, Sven Schreiber wrote:
>> Sorry, to avoid confusion: According to the doc there is
currently
>> already the possibility to write "null" for the specification types
>> 1-3. (For spec type 0 it's already optional.) What I mean is if this
>> explicit "null" could also be made optional.
>
> That would be a bit awkward to arrange. [...]
OK, not really all that awkward. In git we now support dropping the
"theta" (hyperparameter initialization) argument to mds() and mdsl()
altogether, meaning that the initialization is automatic (as if you gave
"null" for the argument).
Could it be that the "almonp" variant was overlooked here?
I have a case here with a term mdsl(dIPL, "almonp", theta1) which works
OK like this, but which fails if I remove the explicit 'theta1' in there.
With the other spec cases it works fine.
thanks,
sven
2:01 p.m.
On Fri, 6 Oct 2017, Sven Schreiber wrote:
Am 14.09.2017 um 22:09 schrieb Allin Cottrell:
> On Thu, 14 Sep 2017, Allin Cottrell wrote:
>
>> On Wed, 13 Sep 2017, Sven Schreiber wrote:
>>> Sorry, to avoid confusion: According to the doc there is currently
>>> already the possibility to write "null" for the specification types
1-3.
>>> (For spec type 0 it's already optional.) What I mean is if this explicit
>>> "null" could also be made optional.
>>
>> That would be a bit awkward to arrange. [...]
>
> OK, not really all that awkward. In git we now support dropping the "theta"
> (hyperparameter initialization) argument to mds() and mdsl() altogether,
> meaning that the initialization is automatic (as if you gave "null" for the
> argument).
Could it be that the "almonp" variant was overlooked here?
I have a case here with a term mdsl(dIPL, "almonp", theta1) which works OK
like this, but which fails if I remove the explicit 'theta1' in there.
With the other spec cases it works fine.
The point here is that as things stand we're not willing to guess how
many terms you want in your Almon polynomial (while with NEAlmon and
beta there's a clear presumption of two parameters as default).
Allin
2:09 p.m.
Am 06.10.2017 um 16:01 schrieb Allin Cottrell:
On Fri, 6 Oct 2017, Sven Schreiber wrote:
> Could it be that the "almonp" variant was overlooked
here?
>
> I have a case here with a term mdsl(dIPL, "almonp", theta1) which
> works OK like this, but which fails if I remove the explicit 'theta1'
> in there.
> With the other spec cases it works fine.
The point here is that as things stand we're not willing to guess how
many terms you want in your Almon polynomial (while with NEAlmon and
beta there's a clear presumption of two parameters as default).
Right, that's reasonable of course!
thanks,
sven
2394
days inactive
2418
days old
33 comments
2 participants
participants (2)
-
Allin Cottrell -
Sven Schreiber