8 p.m.
Dear all,
for a weekly data set, the accessor $obsmicro returns the day instead of
the week. I would have expected that for this frequency the
"micro"-frequency should refer to the calendar week. See here:
<hansl>
open nysewk.gdt -q
series obsmin = $obsminor
series obsmic = $obsmicro # referece to day instead of week
print close obsmin obsmic -o
</hansl>
This is with latest git on ubuntu.
Best,
Artur
1:11 p.m.
Am 14.03.2019 um 21:00 schrieb Artur Tarassow:
Dear all,
for a weekly data set, the accessor $obsmicro returns the day instead
of the week.
Yes, that is according to the spec (=expected).
I would have expected that for this frequency the
"micro"-frequency
should refer to the calendar week.
Well, apart from the fact that it is documented (see above), the ISO
date format and general usage has year/month/day. So week would be
another animal.
I know in Germany we have these "KW" (calendar week) numbers, but I'm
not sure if something like this is usual internationally.
Obviously one could write a hansl function to count and produce the week
numbers. Perhaps by cumulating the output of the seasonals() function or
something. But of course that's more complicated.
cheers,
sven
7:28 a.m.
Am 15.03.19 um 14:11 schrieb Sven Schreiber:
Am 14.03.2019 um 21:00 schrieb Artur Tarassow:
> Dear all,
>
> for a weekly data set, the accessor $obsmicro returns the day instead
> of the week.
Yes, that is according to the spec (=expected).
Thanks for clarification, Sven.
> I would have expected that for this frequency the
"micro"-frequency
> should refer to the calendar week.
Well, apart from the fact that it is documented (see above), the ISO
date format and general usage has year/month/day. So week would be
another animal.
True, in this case.
I know in Germany we have these "KW" (calendar week)
numbers, but I'm
not sure if something like this is usual internationally.
Obviously one could write a hansl function to count and produce the week
numbers. Perhaps by cumulating the output of the seasonals() function or
something. But of course that's more complicated.
Indeed, seems like I need to write a my own function for this ;-)
Thanks,
Artur
4:07 p.m.
Am 16.03.2019 um 08:28 schrieb Artur Tarassow:
Am 15.03.19 um 14:11 schrieb Sven Schreiber:
> Am 14.03.2019 um 21:00 schrieb Artur Tarassow:
>> for a weekly data set, the accessor $obsmicro returns the day
>> instead of the week.
>
> Yes, that is according to the spec (=expected).
Thanks for clarification, Sven.
Actually, thinking some more about it I'm not sure it really is
documented (and perhaps not even explicitly intended). I'm also noticing
that adding periodic dummies (in the GUI) and the seasonals() function
in a weekly dataset work just fine, so in the background there is some
week indicator.
>> I would have expected that for this frequency the
"micro"-frequency
>> should refer to the calendar week.
Given the periodicity 52 perhaps the
"minor" frequency would make even
more sense, thinking about a year:week structure.
Indeed, seems like I need to write a my own function for this ;-)
Here's an idea (relatively brute force):
<hansl>
open nysewk.gdt
list s = seasonals() # gives S1...S52
series obsweek = NA
loop i=1..$pd -q
obsweek = (S$i == 1) ? i : obsweek
endloop
smpl full
</hansl>
cheers
sven
11:59 a.m.
Am 16.03.19 um 17:07 schrieb Sven Schreiber:
Am 16.03.2019 um 08:28 schrieb Artur Tarassow:
> Am 15.03.19 um 14:11 schrieb Sven Schreiber:
>> Am 14.03.2019 um 21:00 schrieb Artur Tarassow:
>>> for a weekly data set, the accessor $obsmicro returns the day
>>> instead of the week.
>>
>> Yes, that is according to the spec (=expected).
>
> Thanks for clarification, Sven.
>
Actually, thinking some more about it I'm not sure it really is
documented (and perhaps not even explicitly intended). I'm also
noticing that adding periodic dummies (in the GUI) and the seasonals()
function in a weekly dataset work just fine, so in the background
there is some week indicator.
Ohh, I totally forgot about the seasonals() functions. Indeed very useful!
>>> I would have expected that for this frequency the
"micro"-frequency
>>> should refer to the calendar week.
Given the periodicity 52 perhaps the "minor" frequency would make even
more sense, thinking about a year:week structure.
Yes, in case one works with
weekly data, the "minor" frequency should be
the calendar week. Also, I am not sure what the value added of a
"micro" frequency would be in this case. So "year:week" would be
totally
sufficient.
> Indeed, seems like I need to write a my own function for this
;-)
Here's an idea (relatively brute force):
<hansl>
open nysewk.gdt
list s = seasonals() # gives S1...S52
series obsweek = NA
loop i=1..$pd -q
obsweek = (S$i == 1) ? i : obsweek
endloop
smpl full
</hansl>
Thanks,
Artur
7:53 p.m.
>>>> I would have expected that for this frequency the
>>>> "micro"-frequency should refer to the calendar week.
> Given the periodicity 52 perhaps the "minor" frequency would make
> even more sense, thinking about a year:week structure.
Yes, in case one works with weekly data, the "minor" frequency should
be the calendar week. Also, I am not sure what the value added of a
"micro" frequency would be in this case. So "year:week" would be
totally sufficient.
Just as a follow up: Python's Pandas library also keeps the structure
<year:month:day> for weekly data. Also <pandas.DatetimeIndex.freq>
involves a series with 52 unique discrete values exactly corresponding
to what gretl's seasonals() function returns (even though you get a list
of series here). So I guess things are fine as they are in gretl ;-)
Best,
Artur
9:57 a.m.
Am 18.03.2019 um 20:53 schrieb Artur Tarassow:
Just as a follow up: Python's Pandas library also keeps the
structure
<year:month:day> for weekly data. Also <pandas.DatetimeIndex.freq>
involves a series with 52 unique discrete values exactly corresponding
to what gretl's seasonals() function returns (even though you get a list
of series here). So I guess things are fine as they are in gretl ;-)
Well, seasonals() is helpful but the large collection of dummies is
certainly no week indicator. Maybe something like my workaround function
could be added to the extra.zip add-on. (If the seasonals are created in
a function they wouldn't be visible and not be flooding the active dataset.)
To be explicit, this is what I mean - Artur, perhaps you could test it
also with a dataset that doesn't start at week 1?
<hansl>
function series obsweek(void)
if $pd != 52
funcerr "Not a weekly dataset"
endif
list s = seasonals() # gives S1...S52
series obsweek = NA
loop i=1..$pd -q
obsweek = (S$i == 1) ? i : obsweek
endloop
setinfo obsweek --description="Week indicator"
setinfo obsweek --discrete # doesn't work !?
return obsweek
end function
</hansl>
cheers
sven
4:21 p.m.
On Fri, 15 Mar 2019, Sven Schreiber wrote:
Am 14.03.2019 um 21:00 schrieb Artur Tarassow:
> Dear all,
>
> for a weekly data set, the accessor $obsmicro returns the day instead of
> the week.
Yes, that is according to the spec (=expected).
> I would have expected that for this frequency the "micro"-frequency should
> refer to the calendar week.
Well, apart from the fact that it is documented (see above), the ISO date
format and general usage has year/month/day. So week would be another animal.
I know in Germany we have these "KW" (calendar week) numbers, but I'm not
sure if something like this is usual internationally.
See https://en.wikipedia.org/wiki/ISO_week_date
It's not trivial to come up with a coherent week numbering system but
I guess we could implement ISO 8601 week numbers (in some form) if
there's a real use case for them.
Allin
4:33 p.m.
Am 19.03.2019 um 17:21 schrieb Allin Cottrell:
See https://en.wikipedia.org/wiki/ISO_week_date
It's not trivial to come up with a coherent week numbering system but I
guess we could implement ISO 8601 week numbers (in some form) if there's
a real use case for them.
Earlier in this thread we noted that gretl already must have week
numbers internally, because creation of seasonal dummies works.
About the use case, I'll leave most of that to Artur who raised the
issue, but I guess in principle sometimes one might want to exclude the
Christmas weeks, or add a dummy for them, or something like that.
Again, perhaps having it in 'extra' instead of core gretl would be enough.
cheers
sven
5:02 p.m.
On Tue, 19 Mar 2019, Sven Schreiber wrote:
Am 19.03.2019 um 17:21 schrieb Allin Cottrell:
> See https://en.wikipedia.org/wiki/ISO_week_date
>
> It's not trivial to come up with a coherent week numbering system but I
> guess we could implement ISO 8601 week numbers (in some form) if there's
> a real use case for them.
Earlier in this thread we noted that gretl already must have week
numbers internally, because creation of seasonal dummies works.
Well, it sort-of works. The weeks won't be the same as in ISO 8601. An
obsweek series based on gretl's weekly seasonals will just repeat
1,2,...,52 across the observations in the dataset, starting at 1
without regard to the starting date (if the observations are dated).
And each pd=52 "year" has exactly 52 weeks, so it's not equivalent to
a real calendar year in either starting-point or length.
Example:
<hansl>
set verbose off
function void make_obsweek(series *ow)
if $pd != 52
funcerr "Not a weekly dataset"
endif
list s = seasonals() # gives S1...S52
ow = NA
loop i=1..$pd -q
ow = (S$i == 1) ? i : ow
endloop
setinfo ow --description="Week indicator"
setinfo ow --discrete
end function
nulldata 120
# start in the first week of April, 2016
setobs 52 2016-04-04
series obsweek
make_obsweek(&obsweek)
eval getinfo(obsweek)
print -o
</hansl>
Allin
6:26 p.m.
On Tue, 19 Mar 2019, Sven Schreiber wrote:
Am 18.03.2019 um 20:53 schrieb Artur Tarassow:
> Just as a follow up: Python's Pandas library also keeps the structure
> <year:month:day> for weekly data. Also <pandas.DatetimeIndex.freq>
> involves a series with 52 unique discrete values exactly corresponding
> to what gretl's seasonals() function returns (even though you get a list
> of series here). So I guess things are fine as they are in gretl ;-)
Maybe so, but weeks defined in that way will not keep in step with the
true calendar and hence will not serve as true seasonals -- though if
the time series is not too long the approximation may be acceptable.
Anyway, in git we now have the function isoweek() which gives the ISO
8601 week number for given year, month and day. Not documented yet but
here's an example (with bonus printing of the "ISO week date" of each
observation):
<hansl>
nulldata 120
setobs 52 2016-04-04
series obsweek = isoweek($obsmajor, $obsminor, $obsmicro)
series isowkdate = index
strings S = array($nobs)
loop i=1..$nobs -q
S[i] = sprintf("%d-W%02d-1", $obsmajor[i], obsweek[i])
endloop
stringify(isowkdate, S)
print obsweek isowkdate -o
</hansl>
Allin
9:27 p.m.
On Tue, 19 Mar 2019, Sven Schreiber wrote:
<hansl>
function series obsweek(void)
if $pd != 52
funcerr "Not a weekly dataset"
endif
list s = seasonals() # gives S1...S52
series obsweek = NA
loop i=1..$pd -q
obsweek = (S$i == 1) ? i : obsweek
endloop
setinfo obsweek --description="Week indicator"
setinfo obsweek --discrete # doesn't work !?
return obsweek
end function
</hansl>
In general there can be two sorts of weekly data in gretl: one
that's just weeks 1 to N without any calendar dates attached, and
another that is indexed by an actual date for the start of each
week.
Question: now that we have the isoweek() function we could redefine
seasonals() to use ISO 8601 week numbers in the second case rather
than just repeating 1 to 52 without regard to the calendar -- would
that be worthwhile?
(If the answer is No, that's fine by me!)
Allin
7:02 a.m.
Am 19.03.2019 um 22:27 schrieb Allin Cottrell:
Question: now that we have the isoweek() function we could redefine
seasonals() to use ISO 8601 week numbers in the second case rather than
just repeating 1 to 52 without regard to the calendar -- would that be
worthwhile?
Probably yes. I've looked more closely at the nysewk.gdt dataset and
added seasonal (periodic) dummies again. The series starts in 1966,
first date 1966-01-05. Scrolling down to the end of the series, the
dummy for the first week of the year has the last entry of 1 for the
date 2005-11-16. I guess that's not really usable.
thanks
sven
10 p.m.
On Wed, Mar 20, 2019 at 3:02 AM Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 19.03.2019 um 22:27 schrieb Allin Cottrell:
>
> Question: now that we have the isoweek() function we could redefine
> seasonals() to use ISO 8601 week numbers in the second case rather than
> just repeating 1 to 52 without regard to the calendar -- would that be
> worthwhile?
Probably yes. I've looked more closely at the nysewk.gdt dataset and
added seasonal (periodic) dummies again. The series starts in 1966,
first date 1966-01-05. Scrolling down to the end of the series, the
dummy for the first week of the year has the last entry of 1 for the
date 2005-11-16. I guess that's not really usable.
In git and snapshots weekly seasonals() now use ISO 8601 week
numbers when the data are dated.
Allin
6:53 a.m.
Hi, sorry for the late reply.
Am 20.03.19 um 08:02 schrieb Sven Schreiber:
Am 19.03.2019 um 22:27 schrieb Allin Cottrell:
>
> Question: now that we have the isoweek() function we could redefine
> seasonals() to use ISO 8601 week numbers in the second case rather than
> just repeating 1 to 52 without regard to the calendar -- would that be
> worthwhile?
Probably yes. I've looked more closely at the nysewk.gdt dataset and
added seasonal (periodic) dummies again. The series starts in 1966,
first date 1966-01-05. Scrolling down to the end of the series, the
dummy for the first week of the year has the last entry of 1 for the
date 2005-11-16. I guess that's not really usable.
Thanks for the isoweek() function, Allin, which is really helpful.
I totally agree with Sven. Especially, given the name of the function
'seasonals', I guess the user would expect getting a series indicating
'correct' calendar weeks.
Best,
Artur
7:02 a.m.
Am 21.03.19 um 23:00 schrieb Cottrell, Allin:
On Wed, Mar 20, 2019 at 3:02 AM Sven Schreiber
<svetosch(a)gmx.net> wrote:
>
> Am 19.03.2019 um 22:27 schrieb Allin Cottrell:
>>
>> Question: now that we have the isoweek() function we could redefine
>> seasonals() to use ISO 8601 week numbers in the second case rather than
>> just repeating 1 to 52 without regard to the calendar -- would that be
>> worthwhile?
>
> Probably yes. I've looked more closely at the nysewk.gdt dataset and
> added seasonal (periodic) dummies again. The series starts in 1966,
> first date 1966-01-05. Scrolling down to the end of the series, the
> dummy for the first week of the year has the last entry of 1 for the
> date 2005-11-16. I guess that's not really usable.
In git and snapshots weekly seasonals() now use ISO 8601 week
numbers when the data are dated.
Just saw this, Allin. Thank you, this is great!
Artur
1856
days inactive
1864
days old
15 comments
4 participants
participants (4)
-
Allin Cottrell -
Artur Tarassow -
Cottrell, Allin
-
Sven Schreiber