Am 19.08.2013 21:14, schrieb Allin Cottrell:
On Mon, 19 Aug 2013, Sven Schreiber wrote:
> well, not Alfred himself, but "his" data -- I'm talking about the
real-time
> database in St. Louis, of course.
>
...
(But, agreed, it's not nice to have to pre-process the input.)
Status update: For now it still seems necessary to pre-process the
input. I did that with Python in a 'foreign' block within the gretl
script, and after some trial-and-error passing around the correctly
formatted path strings it works for me as a temporary solution. I will
send the code when I have wrapped the stuff in functions as far as possible.
Here's how I transformed the Alfred input file (of course I may have
overlooked some nicer solutions):
As you know, the publication date columns (realtime_start_date and
realtime_end_date) come in daily ISO format %Y-%m-%d. But I need to do
an earlier/later filtering operation on them. So I stripped the hyphens
to get pure numbers.
(BTW: I tried to do the same thing with the --tkey column
"observation_date" and specify the format there with
'--time="%Y%m%d"'
[and also "%Y %m %d"], but that produced an error. Expected or a bug?)
I also changed the missing value code in the realtime_end_date from "."
to the number 99999999 for the same reason, to make numerical comparison
possible.
In order to avoid this preprocessing, it would be great if it were
possible to use a column with daily ISO dates in a filtering comparison
directly, as in:
'--filter="realtime_start_date<=20130820"' (or ... <=2013-08-20)
Maybe accompanied with a new option to declare which columns should be
treated as holding ISO-date data:
'--isodates=realtime_start_date,realtime_end_date'
And finally, a wishlist for 'join' options. These missing features can
be worked around, but it would be easier if they existed:
1) '--filter' with two conditions instead of just one; in the realtime
context the natural application is to do a "within" check:
realtime_start_date <= somedate <= realtime_end_date
2) '--aggr="seq:last"', to get the last matching item, where the number
of that last item can vary between different key-matched sequences.
Thanks,
Sven