2:29 a.m.
In current CVS (and Windows snapshot) there's an Open Document
Spreadsheet importer
/File/Open data/Import/Open Document...
If people could test it, that would be very helpful. If it
crashes all over the place we can hide it for the forthcoming
release, otherwise leave it visible.
It does not currently have handling for string-coded variables,
only numeric data. It should be able to extract time-series
information from dates/periods in the first column reasonably
well.
Allin.
--
Allin Cottrell
Department of Economics
Wake Forest University, NC
5:59 p.m.
Allin Cottrell schrieb:
In current CVS (and Windows snapshot) there's an Open Document
Spreadsheet importer
/File/Open data/Import/Open Document...
If people could test it, that would be very helpful. If it
crashes all over the place we can hide it for the forthcoming
release, otherwise leave it visible.
It does not currently have handling for string-coded variables,
only numeric data. It should be able to extract time-series
information from dates/periods in the first column reasonably
well.
If you mean it doesn't read in the variable labels then I can confirm
that. (Instead it generically names the variables as v1, v2, etc.)
But hey, it's working! So thanks a lot for the quick implementation. I
will shortly close the corresponding feature request issue on
sourceforge, or let's say change it to ask for the capability of reading
in the variable names. (BTW, I will also close the Excel 2003 issue soon
since it appears to be a non-issue, at least nobody could give a failing
example, right?)
Some comments though:
1) why only an entry in "open data", not in "append data" as well?
(BTW, I would suggest to match the ordering of data formats in those two
submenus.)
2) I suggest not adding more features before 1.6.6. Apart from the
possibility of breaking stuff by late additions, new features often
require a translation update; and I for example was hoping that the
German translation for 1.6.6 is settled by now.
3) Is there a potential issue with locale and decimal separator settings
differing between Openoffice and gretl?
Thanks,
Sven
7:22 p.m.
On Wed, 7 Nov 2007, Sven Schreiber wrote:
Allin Cottrell schrieb:
> It does not currently have handling for string-coded variables,
> only numeric data. It should be able to extract time-series
> information from dates/periods in the first column reasonably
> well.
>
If you mean it doesn't read in the variable labels then I can
confirm that. (Instead it generically names the variables as v1,
v2, etc.)
No, I don't mean that, and with all the files I've tried
(admittedly, not a huge number) variable names have been read
correctly. If you could send me a copy of a file where the names
are not read, that would be helpful.
1) why only an entry in "open data", not in "append
data" as well?
Yes, it should be under "append" too.
2) I suggest not adding more features before 1.6.6. Apart from
the possibility of breaking stuff by late additions, new
features often require a translation update; and I for example
was hoping that the German translation for 1.6.6 is settled by
now.
Apart from ODS, there have been a few minor fixes lately that
required small changes to translatable strings. I'm updating
gretl.pot in case anyone wants to have another go. But I agree,
no more features at this point.
3) Is there a potential issue with locale and decimal separator
settings differing between Openoffice and gretl?
I don't think so. I believe that the internal numerical value
fields in ODS files are always written using the C locale (though
the displayed version may be locale-specific), and gretl forces
the C locale when reading the files.
Allin.
7:35 p.m.
Allin Cottrell schrieb:
On Wed, 7 Nov 2007, Sven Schreiber wrote:
> Allin Cottrell schrieb:
>> It does not currently have handling for string-coded variables,
>> only numeric data. It should be able to extract time-series
>> information from dates/periods in the first column reasonably
>> well.
>>
> If you mean it doesn't read in the variable labels then I can
> confirm that. (Instead it generically names the variables as v1,
> v2, etc.)
No, I don't mean that, and with all the files I've tried
(admittedly, not a huge number) variable names have been read
correctly. If you could send me a copy of a file where the names
are not read, that would be helpful.
<just sent Allin a file off list where the top two rows are empty, the
third row holds the labels, and then the actual data start at row 4>
Actually, if I tell gretl to start reading at row 2 (instead of 3), I
get a strange error: 'sheet row 2, column 2("const"): expected date
string but found numerical value' (where the row-2/col-2-cell in fact is
empty).
thanks,
sven
10:39 p.m.
On Wed, 7 Nov 2007, Sven Schreiber wrote:
<just sent Allin a file off list where the top two rows are empty,
the
third row holds the labels, and then the actual data start at row 4>
Actually, if I tell gretl to start reading at row 2 (instead of 3), I
get a strange error: 'sheet row 2, column 2("const"): expected date
string but found numerical value' (where the row-2/col-2-cell in fact is
empty).
Thanks for the file, very helpful.
I didn't realize ODS does "number-row-repeated" as well as
"number-column-repeated". The importer in CVS/snapshot should now
work better.
However, on the date strings, I don't really feel like putting
a lot of work into fixing up data that are written back to front!
(These data start in 2006 and end in 1997.) That can be fixed
using sortby() followed by setobs.
Allin.
9:25 a.m.
Allin Cottrell schrieb:
On Wed, 7 Nov 2007, Sven Schreiber wrote:
> <just sent Allin a file off list where the top two rows are empty, the
> third row holds the labels, and then the actual data start at row 4>
>
> Actually, if I tell gretl to start reading at row 2 (instead of 3), I
> get a strange error: 'sheet row 2, column 2("const"): expected date
> string but found numerical value' (where the row-2/col-2-cell in fact is
> empty).
Thanks for the file, very helpful.
I didn't realize ODS does "number-row-repeated" as well as
"number-column-repeated". The importer in CVS/snapshot should now
work better.
Thanks I will test tomorrow (hopefully).
However, on the date strings, I don't really feel like putting
a lot of work into fixing up data that are written back to front!
(These data start in 2006 and end in 1997.) That can be fixed
using sortby() followed by setobs.
Of course, I wasn't complaining about that, it's just the first file I
came across on my disk.
cheers,
sven
7:13 p.m.
I imported some daily daily data into gretl using the ods importer.
Dates were in column one and variable names in row 1. It recognised
my data as a daily time series with 5 days per week and had the
correct start date. Excellent!! However I found a small problem (or
at least it did not do exactly what i wanted). The first few rows of
the ods file were
Date Alpha Beta Gamma Delta
1991-12-30 1.6490 -0.0307 0.5301 -0.0587
1991-12-31 1.6497 -0.0303 0.5303 -0.0588
1992-01-02 1.6510 -0.0273 0.5304 -0.0583
1992-01-03 1.6517 -0.0260 0.5305 -0.0581
1992-01-06 1.6517 -0.0258 0.5305 -0.0582
1992-01-07 1.6515 -0.0245 0.5301 -0.0576
The first of these is a Monday. Note that Wednesday 1992-01-01 was
missing from my data set as this is a bank holiday. Gretl however
inserted a 1 January date and allocated the data from 2 January to 1
January. I assume that gretl decides correctly that the series is
daily, 5 days per week, and starts on 1991-12-30. It then sets up a
list of dates with 5 days per week and allocates the data to those
dates without reference to the dates on the ods file.
Would it be possible for a future version of the ods importer to check
that the dates correspond. If they did not perhaps it could write NAs
in the gretl file against the missing date and then check the next
date.
Best Regards
John
On 08/11/2007, Sven Schreiber <svetosch(a)gmx.net> wrote:
Allin Cottrell schrieb:
> On Wed, 7 Nov 2007, Sven Schreiber wrote:
>
>> <just sent Allin a file off list where the top two rows are empty, the
>> third row holds the labels, and then the actual data start at row 4>
>>
>> Actually, if I tell gretl to start reading at row 2 (instead of 3), I
>> get a strange error: 'sheet row 2, column 2("const"): expected
date
>> string but found numerical value' (where the row-2/col-2-cell in fact is
>> empty).
>
> Thanks for the file, very helpful.
>
> I didn't realize ODS does "number-row-repeated" as well as
> "number-column-repeated". The importer in CVS/snapshot should now
> work better.
Thanks I will test tomorrow (hopefully).
>
> However, on the date strings, I don't really feel like putting
> a lot of work into fixing up data that are written back to front!
> (These data start in 2006 and end in 1997.) That can be fixed
> using sortby() followed by setobs.
>
Of course, I wasn't complaining about that, it's just the first file I
came across on my disk.
cheers,
sven
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
--
John C Frain
Trinity College Dublin
Dublin 2
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
8:17 p.m.
On Thu, 8 Nov 2007, John C Frain wrote:
I imported some daily daily data into gretl using the ods importer.
Dates were in column one and variable names in row 1. It recognised
my data as a daily time series with 5 days per week and had the
correct start date. Excellent!!
So far, so good.
However I found a small problem (or at least it did not do
exactly what i wanted). The first few rows of the ods file were
Date Alpha Beta Gamma Delta
1991-12-30 1.6490 -0.0307 0.5301 -0.0587
1991-12-31 1.6497 -0.0303 0.5303 -0.0588
1992-01-02 1.6510 -0.0273 0.5304 -0.0583
1992-01-03 1.6517 -0.0260 0.5305 -0.0581
1992-01-06 1.6517 -0.0258 0.5305 -0.0582
1992-01-07 1.6515 -0.0245 0.5301 -0.0576
The first of these is a Monday. Note that Wednesday 1992-01-01
was missing from my data set as this is a bank holiday. Gretl
however inserted a 1 January date and allocated the data from 2
January to 1 January. I assume that gretl decides correctly
that the series is daily, 5 days per week, and starts on
1991-12-30. It then sets up a list of dates with 5 days per
week and allocates the data to those dates without reference to
the dates on the ods file.
Yes, that's what happened, though in fact it was not what was
intended. Fixed in CVS and snapshot.
Would it be possible for a future version of the ods importer to
check that the dates correspond. If they did not perhaps it
could write NAs in the gretl file against the missing date and
then check the next date.
The importer should now respect the date labels, provided they're
consistent. It would be possible to insert NAs, but I think in
most cases that's not what one wants. With NAs, you can't do
anything autoregressive. And with daily market data where the NAs
(if inserted) represent non-trading days it's probably legitimate
to treat the data as if there's nothing missing.
Suppose the markets didn't trade on a certain Thursday. Do we say
that Friday's y(-1) is NA, or that it is Wednesday's observation
on y? I suspect most of the time you'd want the latter (and would
be annoyed if gretl stuck in NAs that weren't there in the
original file).
Allin.
9:59 p.m.
Thanks for so quick a solution. I have tried and can confirm that the
ods importer now respects the dates in the ods file. I think that
what you are doing is probably what most people will want. In
Rmetrics it is possible to strip the NAs from a daily series (all 5
days per week) and retain what you are proposing. It is also possible
to add back in the na's to the reduced series to recover the original
5-day per week series with missing values. I have used this facility
to then interpolate for the missing values.
I have also had a look at adding data cells containing =NA() (or #N/A)
opposite the holiday dates in the ods file. These are imported into
gretl as 0. If I replace the =NA() by any string such as NA then that
is also imported as 0. If I just enter the date in the first column
and leave the data entries blank they appear as blank in gretl's |
data | display values | and I presume are then treated as missing by
gretl.
Should the =NA() in ods file be also treated as missing? As one who
uses R it might also be useful to recognise NA as a missing value.
Best regards
John
On 08/11/2007, Allin Cottrell <cottrell(a)wfu.edu> wrote:
On Thu, 8 Nov 2007, John C Frain wrote:
> I imported some daily daily data into gretl using the ods importer.
> Dates were in column one and variable names in row 1. It recognised
> my data as a daily time series with 5 days per week and had the
> correct start date. Excellent!!
So far, so good.
> However I found a small problem (or at least it did not do
> exactly what i wanted). The first few rows of the ods file were
>
> Date Alpha Beta Gamma Delta
> 1991-12-30 1.6490 -0.0307 0.5301 -0.0587
> 1991-12-31 1.6497 -0.0303 0.5303 -0.0588
> 1992-01-02 1.6510 -0.0273 0.5304 -0.0583
> 1992-01-03 1.6517 -0.0260 0.5305 -0.0581
> 1992-01-06 1.6517 -0.0258 0.5305 -0.0582
> 1992-01-07 1.6515 -0.0245 0.5301 -0.0576
>
> The first of these is a Monday. Note that Wednesday 1992-01-01
> was missing from my data set as this is a bank holiday. Gretl
> however inserted a 1 January date and allocated the data from 2
> January to 1 January. I assume that gretl decides correctly
> that the series is daily, 5 days per week, and starts on
> 1991-12-30. It then sets up a list of dates with 5 days per
> week and allocates the data to those dates without reference to
> the dates on the ods file.
Yes, that's what happened, though in fact it was not what was
intended. Fixed in CVS and snapshot.
> Would it be possible for a future version of the ods importer to
> check that the dates correspond. If they did not perhaps it
> could write NAs in the gretl file against the missing date and
> then check the next date.
The importer should now respect the date labels, provided they're
consistent. It would be possible to insert NAs, but I think in
most cases that's not what one wants. With NAs, you can't do
anything autoregressive. And with daily market data where the NAs
(if inserted) represent non-trading days it's probably legitimate
to treat the data as if there's nothing missing.
Suppose the markets didn't trade on a certain Thursday. Do we say
that Friday's y(-1) is NA, or that it is Wednesday's observation
on y? I suspect most of the time you'd want the latter (and would
be annoyed if gretl stuck in NAs that weren't there in the
original file).
Allin.
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
--
John C Frain
Trinity College Dublin
Dublin 2
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
10:54 p.m.
On Thu, 8 Nov 2007, John C Frain wrote:
Thanks for so quick a solution. I have tried and can confirm that
the
ods importer now respects the dates in the ods file. I think that
what you are doing is probably what most people will want. In
Rmetrics it is possible to strip the NAs from a daily series (all 5
days per week) and retain what you are proposing. It is also possible
to add back in the na's to the reduced series to recover the original
5-day per week series with missing values. I have used this facility
to then interpolate for the missing values.
I've been thinking about something along those lines. More on that
later.
I have also had a look at adding data cells containing =NA() (or
#N/A) opposite the holiday dates in the ods file. These are
imported into gretl as 0.
Ah, that's because such cells contain the numeric value 0
internally -- but the text "#N/A" is also stored internally, so we
ought to recognize that. Fixed in CVS.
If I replace the =NA() by any string such as NA then that is
also imported as 0.
Gretl should now recognize the strings
NA, N.A, n.a, N/A, NaN, . and ..
as missing values.
If I just enter the date in the first
column and leave the data entries blank they appear as blank in
gretl's | data | display values | and I presume are then treated
as missing by gretl.
Yes, that's right.
Allin.
6268
days inactive
6269
days old
9 comments
3 participants
participants (3)
-
Allin Cottrell -
John C Frain -
Sven Schreiber