Re: [Gretl-devel] The mysterious stack() function
On Thu, 13 Sep 2018, Sven Schreiber wrote:
Am 11.09.2018 um 15:20 schrieb Allin Cottrell:
> On Tue, 11 Sep 2018, Sven Schreiber wrote:
>
>> Am 10.09.2018 um 21:16 schrieb Allin Cottrell:
>>> On Mon, 10 Sep 2018, Sven Schreiber wrote:
>>> I guess I still have some thoughts and comments about the function
>>> signature and defaults and what kind of text file input arrangement is
>>> supported in detail, but I don't have time right now. Since it's pure
>>> hansl I'm happy to make concrete code proposals later.
>>>
>>> Fine. I now have an improved prototype which I'm happy to supply on
>>> request. It now has some proper error-checking, and it adds "unit"
and
>>> "time" series to the CSV file to make it easier to get gretl to
recognize
>>> the data as a panel.
>>
>> OK, great. On and after Thursday or Friday I should have some time for
>> this. If you're not working on it until then, you might as well send it
>> now.
>
> Alright. Hansl and toy test file attached.
Hmm, actually from the explanation in the guide I had a different kind of
input file in mind, namely something like the following (also attached as a
standalone file):
x1
1965 1970 1975 1980 1985
AR 100.0 110.5 118.7 131.2 160.4
AZ 100.0 104.3 113.8 120.9 140.6
x2
1965 1970 1975 1980 1985
AR 103.0 112.5 111.7 136.2 168.4
AZ 109.0 100.3 110.8 128.9 141.6
Your function fails on this kind of file.
Yes, that's expected. The relevant section of the Guide states that
such a file must be edited, to remove the "extraneous" lines, before
processing via stack() -- although that statement is in a footnote and
may not have the prominence it deserves. In my new, experimental
pstack() function I was making the same assumption: the raw datafile
will have been manually pre-processed to leave just the actual data
blocks.
I have started some string-processing work to handle this, but
I'm
having doubts -- what are we really targeting? It would be helpful
to have a real-world example.
I'd be surprised if it's possible to automate the pre-processing with
any useful generality. But as to what we're targeting, I'm attaching
an example from the IMF (see https://www.imf.org/en/Data , under
"IMF Data Mapper"). IIRC, such files can sometimes have extra lines of
metadata above or below the data block.
Allin
Attachments:
- imf-gdp.csv (text/csv — 989 bytes)