Re: [Gretl-devel] Midas and join (or append)
On Thu, 11 Jan 2018, Riccardo (Jack) Lucchetti wrote:
On Thu, 11 Jan 2018, Sven Schreiber wrote:
> Hi,
>
> I will be working with Midas in gretl again soon, and I've just reread (or
> let's say, re-browsed) the midas_gretl.pdf guide.
> Looking at the section "importation via join", it occurs to me that this is
> quite involved. In contrast, "importation from a database" is very simple.
>
> Shouldn't it be possible to just do something like this:
> <hansl-future>
> open AWM.gdt # quarterly
> join hamilton.gdt EXRITL --compact=spread # monthly to quarterly
> </hansl-future>
>
> or perhaps the 'append' command is better suited than join here? So replace
> the last line above with (which however gets all series, but that's OK):
>
> append hamilton.gdt --spread
I vote for join. IMO, the join command is kind of a relatively unknown gem
that we have and I think your proposal, if implemented, will make it quite
popular among people who work with time series datasets.
I agree that it would be nice to take the opportunity to raise the
profile of "join". But there are two angles here:
1) How do we actually implement this?
2) How do we "package" whatever we implement for users?
On the first point, one option is doing something along the lines
Sven suggested; that is, the internal equivalent of
<hansl>
pd = $pd
store temp.gdt # the current dataset
open hamilton.gdt --preserve # higher-frequency data
dataset compact pd spread
store tempcompact.gdt
open temp.gdt
append tempcompact.gdt
</hansl>
I think this would be relatively straightforward. Internally, we
wouldn't have to store and replace the original data; we can "open"
and compact a dataset such as hamilton.gdt without disturbing the
prior data in userspace.
The other option would be to extend the "join" code itself to handle
compaction on the fly. In a sense this would be cleaner but I'm
pretty sure it would be more complicated; "join" is already quite
complicated.
That leaves the "packaging" question: in principle whatever we come
up with could be presented as as a "join" option even if the
internals are closer to "append".
Allin