Re: [Gretl-devel] grabbing Alfred
On Mon, 19 Aug 2013, Sven Schreiber wrote:
well, not Alfred himself, but "his" data -- I'm talking
about the real-time
database in St. Louis, of course.
I'm working on some functions which would allow to access the Alfred data
directly from the (tab-delimited text) files that you can download there, and
of course I'm running into problems (which is not unexpected). For example,
consider the following attempt to get the latest vintage/publication of each
datapoint from the attached partial file:
<hansl>
nulldata 500 --preserve
setobs 12 1960:01 --time-series # monthly =12 in this example
join INDPRO_excerpt.txt INDPRO --tkey=observation_date
--filter="realtime_end_date==."
</hansl>
There are at least two issues here already:
1) I had to transform the "observation_date" column to match the monthly
frequency in the gretl workfile: Alfred uses the first day of the month to
indicate that month, like "1980-02-01" for February 1980. So I chopped off
the trailing "-01" part of that column to get gretl-compatible monthly date
strings.
No need for that. Use the option --time="%Y-%m-%d" to set the appropriate
time format. (See the current CVS User's Guide.)
2) The above command still produces an error, namely:
"join: missing string in filtering"
I thought it has to do with the dot "." (which indicates NA in Alfred's
text
file output, with Alfred's XLS files it's "#NV"), perhaps it wasn't
recognized by gretl/join as a string constant.
Ah, there's an interesting "gotcha" here. The field "." in
delimited-text
data input is mapped to NA by gretl (as in fact intended by Alfred), so
such observations of realtime_end_date are treated as missing, not as the
string literal ".". Hence the error message about a "missing string".
One
way around that problem would be to introduce a further filtering option
such as
--filter=missing(right-hand-colname)
In the meantime one can replace "." in the input by a dummy string that
doesn't map to NA for gretl, e.g. "x" then do
--filter="realtime_end_date==x"
(But, agreed, it's not nice to have to pre-process the input.)
Allin