9:26 p.m.
I am checking some gretl code against CATS in RATS and using the australian
dataset. When exporting to csv, the variable iau2 only has three digits
after the dot, but the variable has up to four non zero digits after the
dot in gretl and hence I get different results from the exported data than
I do in gretl. Is it a bug?
Best
Andreas
9:34 p.m.
On Tue, 5 Mar 2013, Andreas Noack Jensen wrote:
I am checking some gretl code against CATS in RATS and using the
australian
dataset. When exporting to csv, the variable iau2 only has three digits
after the dot, but the variable has up to four non zero digits after the
dot in gretl and hence I get different results from the exported data than
I do in gretl. Is it a bug?
Looks like it. I'll take a look at the relevant code shortly.
Allin
10:08 p.m.
On Tue, 5 Mar 2013, Allin Cottrell wrote:
On Tue, 5 Mar 2013, Andreas Noack Jensen wrote:
> I am checking some gretl code against CATS in RATS and using the australian
> dataset. When exporting to csv, the variable iau2 only has three digits
> after the dot, but the variable has up to four non zero digits after the
> dot in gretl and hence I get different results from the exported data than
> I do in gretl. Is it a bug?
Looks like it. I'll take a look at the relevant code shortly.
In the meantime, you can use
set csv_digits 18
and force the number of digits in the csv file.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
1:14 a.m.
On Tue, 5 Mar 2013, Riccardo (Jack) Lucchetti wrote:
On Tue, 5 Mar 2013, Allin Cottrell wrote:
> On Tue, 5 Mar 2013, Andreas Noack Jensen wrote:
>
>> I am checking some gretl code against CATS in RATS and using the
>> australian
>> dataset. When exporting to csv, the variable iau2 only has three digits
>> after the dot, but the variable has up to four non zero digits after the
>> dot in gretl and hence I get different results from the exported data than
>> I do in gretl. Is it a bug?
>
> Looks like it. I'll take a look at the relevant code shortly.
In the meantime, you can use
set csv_digits 18
and force the number of digits in the csv file.
Hmm, yes, we need to document that.
Anyway, I found the bug. We were trying to be a bit too clever in
saving bytes in CSV output, and failed (occasionally) in quite a
subtle way. Fixed in CVS; snapshots will follow tomorrow.
Allin
5:46 a.m.
Thank you, both of you.
2013/3/6 Allin Cottrell <cottrell(a)wfu.edu>
On Tue, 5 Mar 2013, Riccardo (Jack) Lucchetti wrote:
> On Tue, 5 Mar 2013, Allin Cottrell wrote:
>
>> On Tue, 5 Mar 2013, Andreas Noack Jensen wrote:
>>
>>> I am checking some gretl code against CATS in RATS and using the
>>> australian
>>> dataset. When exporting to csv, the variable iau2 only has three digits
>>> after the dot, but the variable has up to four non zero digits after
the
>>> dot in gretl and hence I get different results from the exported data
than
>>> I do in gretl. Is it a bug?
>>
>> Looks like it. I'll take a look at the relevant code shortly.
>
> In the meantime, you can use
>
> set csv_digits 18
>
> and force the number of digits in the csv file.
Hmm, yes, we need to document that.
Anyway, I found the bug. We were trying to be a bit too clever in
saving bytes in CSV output, and failed (occasionally) in quite a
subtle way. Fixed in CVS; snapshots will follow tomorrow.
Allin
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
--
Med venlig hilsen
Andreas Noack Jensen
2:35 p.m.
On Tue, 5 Mar 2013, Allin Cottrell wrote:
On Tue, 5 Mar 2013, Riccardo (Jack) Lucchetti wrote:
> On Tue, 5 Mar 2013, Allin Cottrell wrote:
>
>> On Tue, 5 Mar 2013, Andreas Noack Jensen wrote:
>>
>>> I am checking some gretl code against CATS in RATS and using the
>>> australian
>>> dataset. When exporting to csv, the variable iau2 only has three digits
>>> after the dot, but the variable has up to four non zero digits after the
>>> dot in gretl and hence I get different results from the exported data
>>> than
>>> I do in gretl. Is it a bug?
>>
>> Looks like it. I'll take a look at the relevant code shortly.
>
> In the meantime, you can use
>
> set csv_digits 18
>
> and force the number of digits in the csv file.
Hmm, yes, we need to document that.
I was about to, but then I realised that maybe at this point we may want
to revisit this setting to make it more intuitive. My understanding of the
way it works at present is: if csv_digits is n, then see what happens with
n decimal digits after the decimal separator. If for a series the last m
digits are all zeros, then use n-m digits for that variable. So it would
seem that csv_digits is an _upper_ bound. However, if you set it to some
large value (like 18), then machine precision kicks in and the last digits
are not 0 because of numerical noise, so no truncation occurs and you
effectively get n digits after the decimal separator for all variables.
This does not strike me as particularly intuitive. I guess that most users
would interpret a setting like
set cvs_digits 8
as "use 8 digits all the time, come what may". Allin, I imagine you have
a good reason against this, but I fail to see it right now.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:05 p.m.
On Wed, 6 Mar 2013, Riccardo (Jack) Lucchetti wrote:
[re. the "set" variable named "csv_digits", which is not
currently documented]
I was about to [document it], but then I realised that maybe
at this point we may want to revisit this setting to make it
more intuitive. My understanding of the way it works at
present is: if csv_digits is n, then see what happens with n
decimal digits after the decimal separator. If for a series
the last m digits are all zeros, then use n-m digits for
that variable. So it would seem that csv_digits is an
_upper_ bound. However, if you set it to some large value
(like 18), then machine precision kicks in and the last
digits are not 0 because of numerical noise, so no
truncation occurs and you effectively get n digits after the
decimal separator for all variables.
This does not strike me as particularly intuitive. I guess
that most users would interpret a setting like
set cvs_digits 8
as "use 8 digits all the time, come what may". Allin, I
imagine you have a good reason against this, but I fail to
see it right now.
Here's what the _intent_ is (we need to do some more checking
to see if it's properly implemented, but I think it is):
1) If you don't set "csv_digits", then gretl prints CSV values
using an algorithm designed to preserve precision to a
reasonable degree while producing a reasonably elegant and
easily readable output file (details below).
2) If you set a value for "csv_digits" then gretl doesn't try
to do anything clever, it just prints CSV values to the
specified number of significant digits (not decimal places).
Note that we use "%g" conversion, so trailing zeros are
dropped.
Details on the default algorithm: We check each series to
determine if it can be printed in "%f" format, with a
reasonably small uniform number of decimal places, without
loss of precision.[*] This is often the case for "primary"
data (as opposed to logs and such). If so, do it; if not,
print to 12 significant figures using "%g" format. The 12 here
is somewhat arbitrary, but the idea is to preserve reasonable
precision while avoiding printing "trailing junk" that is just
an artifact of the finite precision of 64-bit floating point
numbers. (You can see such junk if you open australia.gdt, do
"set csv digits 18", store as CSV, and open the CSV file in a
text editor.)
[*] The bug that Andreas exposed was a flaw in this procedure,
now fixed.
Allin
3:36 p.m.
On Wed, 6 Mar 2013, Allin Cottrell wrote:
2) If you set a value for "csv_digits" then gretl
doesn't try to do anything
clever, it just prints CSV values to the specified number of significant
digits (not decimal places). Note that we use "%g" conversion, so trailing
zeros are dropped.
We could use "%#g" conversion to preserve trailing zeros in
CSV output when csv_digits is set, but I don't really see the
advantage in that. Trailing zeros are "significant digits" in
a mathematical sense but they don't have any significance to a
computer reading numerical input from text (as many trailing
zeros as the double precision floating-point type supports are
implicit in a numeric string such as "1.52").
Allin
4071
days inactive
4072
days old
7 comments
3 participants
participants (3)
-
Allin Cottrell -
Andreas Noack Jensen -
Riccardo (Jack) Lucchetti