9:20 a.m.
Hi,
Gretl let me down :( Full of enthusiasm I praised Gretl to my students, but they
couldn't even import my csv files with their current release! So I got me their
version 2016a and did some experiments. I found out the following:
The import dialogues for the German and English versions are different. If you set up the
right decimal separator in the German version, everything works fine. If you got it wrong
(which my students did), the dialogue NEVER reappears! So you have no chance to correct
your mistake and you will not be able to load any file with different decimal settings.
Could you please correct this and let the import dialogue reappear? If a user has
different sources with different separator rules, he should be able to adjust the settings
for each file. Or is there any "settings" option at another place that I am not
aware of?
Current workaround is to change the language setting back to English, restart and set it
back to German. Then the dialogue will reappear.
Thanks for your help!
Kind regards,
Thorsten
Prof. Dr. Thorsten Wingenroth
Professor für Lehraufgaben BWL-Bank
Fakultät Wirtschaft | Studienzentrum Finanzwirtschaft (Center of Finance)
Duale Hochschule Baden-Württemberg Stuttgart
Baden-Wuerttemberg Cooperative State University Stuttgart
Herdweg 18 | 70174 Stuttgart
Fon +49 711 1849-766
thorsten.wingenroth@dhbw-stuttgart.de<mailto:thorsten.wingenroth@dhbw-stuttgart.de>
| http://www.dhbw-stuttgart.de/bank
Master in Business Management - Banking & Finance
http://www.dhbw.de/master-finance
9:11 p.m.
On Mon, 1 Feb 2016, Wingenroth, Thorsten wrote:
Gretl let me down :( Full of enthusiasm I praised Gretl to my
students, but they couldn't even import my csv files with their
current release!
Wat do your CSV files look like? Could you send a sample (either on
or off-list), please?
The import dialogues for the German and English versions are
different. If you set up the right decimal separator in the German
version, everything works fine.
You only get a CSV options dialog if you're opening an
already-opened file off the recent files list. That's actually a
mistake, the dialog should never be shown nowadays, since gretl
is cleverer than most users at figuring out the characteristics of
a CSV file. But if your files are an exception we need to
reconsider...
(There's one other context where the dialog should be, and is,
shown, and that's on exporting CSV data.)
Allin Cottrell
2:51 a.m.
Hi Allin,
sorry, I wanted to attach them...
Kind regards,
Thorsten
-----Ursprüngliche Nachricht-----
Von: gretl-users-bounces(a)lists.wfu.edu [mailto:gretl-users-bounces@lists.wfu.edu] Im
Auftrag von Allin Cottrell
Gesendet: Dienstag, 2. Februar 2016 03:12
An: Gretl list
Betreff: Re: [Gretl-users] Input of csv data - does the German version work as intended?
On Mon, 1 Feb 2016, Wingenroth, Thorsten wrote:
Gretl let me down :( Full of enthusiasm I praised Gretl to my
students, but they couldn't even import my csv files with their
current release!
Wat do your CSV files look like? Could you send a sample (either on or off-list), please?
The import dialogues for the German and English versions are
different. If you set up the right decimal separator in the German
version, everything works fine.
You only get a CSV options dialog if you're opening an already-opened file off the
recent files list. That's actually a mistake, the dialog should never be shown
nowadays, since gretl is cleverer than most users at figuring out the characteristics of a
CSV file. But if your files are an exception we need to reconsider...
(There's one other context where the dialog should be, and is, shown, and that's
on exporting CSV data.)
Allin Cottrell
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
9:14 a.m.
On Tue, 2 Feb 2016, Wingenroth, Thorsten wrote:
Hi Allin,
sorry, I wanted to attach them...
Thanks, Thorsten. For reference, these files use ',' and decimal
separator and ';' as field delimiter (OK). They also use '.' as
thousands marker and as the delimiter within daily dates (and into the
bargain the dates are reversed). So for example:
<csv>
date;Erster;Hoch;Tief;Schlusskurs;Stuecke;Volumen
23.02.2015;71,16;71,24;70,6;70,91;172.975;12.251.361
20.02.2015;70,55;70,92;70,22;70,62;752.159;53.136.228
19.02.2015;70,74;71,06;69,93;70,73;576.821;40.735.744
...
</csv>
Gretl running in English gets everything right automatically, but the
importation fails when running in German -- in fact, in any locale
that uses ',' as decimal character and with "Use locale setting for
decimal point" selected in gretl's Preferences.
A workaround for opening such files in German is to deselect "Use
local setting for decimal point". However, it should work either way;
we'll see what we can do about that. The problem seems to lie in the
handling of the thousands separator: if I delete those '.' characters
from one of your files gretl reads it fine even when in decimal comma
mode.
Allin
3:50 p.m.
On Tue, 2 Feb 2016, Allin Cottrell wrote:
<csv>
date;Erster;Hoch;Tief;Schlusskurs;Stuecke;Volumen
23.02.2015;71,16;71,24;70,6;70,91;172.975;12.251.361
20.02.2015;70,55;70,92;70,22;70,62;752.159;53.136.228
19.02.2015;70,74;71,06;69,93;70,73;576.821;40.735.744
...
</csv>
Gretl running in English gets everything right automatically, but
the importation fails when running in German -- in fact, in any
locale that uses ',' as decimal character and with "Use locale
setting for decimal point" selected in gretl's Preferences.
This is now fixed in git and snapshots.
When gretl realized that '.' must be a thousands marker, she revised
her opinion about ',', now taking it to be the decimal separator. The
consequences of this switch were correctly handled for the case where
the local decimal separator is dot, but not otherwise. Now they're
handled in both cases.
Allin
4:19 p.m.
Once again, a problem for data analysis arises when
someone insists that at **COMMA** separated values
file should be allowed to use commas for some other purpose.
Stick to the RFC and these problems go away:
https://tools.ietf.org/html/rfc4180
I understand why it was done, but I think the decision
to handle non-conforming files that claim to be CSV
was more politically correct than scientifically correct.
For CSV data exchange to be internationally feasible,
we need to stick to an international standard.
Cheers,
Alan Isaac
5:11 p.m.
On Tue, 2 Feb 2016, Alan G Isaac wrote:
Once again, a problem for data analysis arises when
someone insists that at **COMMA** separated values
file should be allowed to use commas for some other purpose.
Stick to the RFC and these problems go away:
https://tools.ietf.org/html/rfc4180
I understand why it was done, but I think the decision
to handle non-conforming files that claim to be CSV
was more politically correct than scientifically correct.
For CSV data exchange to be internationally feasible,
we need to stick to an international standard.
You have a strong figure in your corner -- at least against the
decimal comma for anything other than display purposes -- namely
Jack Lucchetti, but I'm afraid I'm a wretched compromiser.
The label "CSV" has become detached from its literal meaning, and
now designates pretty much any column-oriented "delimited text" data
file, where the column delimiter may be comma, tab, space or
semicolon and the decimal separator may be dot or comma.
As a coder, I find the problem of interpreting such files quite an
interesting one, up to a point. The cases of decimal-dot plus comma
delimiter and decimal-comma plus semicolon delimiter are quite
easily figured out. Things get a lot more complicated when
thousands-separators are introduced: it is totally crazy to put such
decorations in files that are supposed to be computer-readable. But
there's a lot of it out there, so we try our best to come up with an
algorithm that will extract the signal from the noise.
Allin
5:25 p.m.
On Tue, 2 Feb 2016, Allin Cottrell wrote:
You have a strong figure in your corner -- at least against the
decimal
comma for anything other than display purposes -- namely Jack Lucchetti,
but I'm afraid I'm a wretched compromiser.
:D
I think I'll just keep quiet.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
4:23 a.m.
El 02/02/16 a las 22:19, Alan G Isaac escribió:
Once again, a problem for data analysis arises when
someone insists that at **COMMA** separated values
file should be allowed to use commas for some other purpose.
Stick to the RFC and these problems go away:
https://tools.ietf.org/html/rfc4180
I understand why it was done, but I think the decision
to handle non-conforming files that claim to be CSV
was more politically correct than scientifically correct.
For CSV data exchange to be internationally feasible,
we need to stick to an international standard.
Cheers,
Alan Isaac
Look at https://en.wikipedia.org/wiki/Decimal_mark
It has a list ofcountries using decimal point, another list with
countries using decimal comma and a nice graph. Half the world uses the
decimal comma.
This strict CSV format cannot be an international standard. It is not
"internationally feasible" because of the ambiguity of the comma. Of
course, someone may pretend to change the official standards for more
than 60 countries, but I think it is easier to relax the CSV definition.
--
Ignacio Díaz-Emparanza
Departamento de Economía Aplicada III (Econometría y Estadística)
Universidad del País Vasco - Euskalherriko Unibertsitatea, UPV/EHU
Tfno: (+34) 94 601 3732
http://www.ehu.eus/ea3
1:37 p.m.
On 2/3/2016 4:23 AM, Ignacio Diaz-Emparanza wrote:
Half the world uses the decimal comma.
This strict CSV format cannot be an international standard. It is not
"internationally feasible" because of the ambiguity of the comma.
This is simply a non-sequitur.
We currently have an unfortunate situation:
some countries have not yet transitioned to a period
for the decimal mark. This means casual communication
is plagued by an ambiguity.
This need not be the case for scientific communication.
There is a standard. Use it. It is really that simple.
Well ... almost: we need some way to communicate adherence to the standard.
That is the role that should be but currently is not played by the .csv extension.
Note that this lack of ambiguity is already present in almost
all computing languages: the dot is the decimal separator.
Everyone needs to bite the bullet and accept that this is
**already** an international standard, with unfortunately poor adherence.
Science is inherently international and needs single standards.
Arguing for the use of the comma is like arguing for the use of the inch.
Cheers,
Alan
4:19 p.m.
"Transitioned..."
Don't ya just *luuuuurve* Americans? ;P
Tidings,
C
On 5 February 2016 at 18:37, Alan G Isaac <alan.isaac(a)gmail.com> wrote:
On 2/3/2016 4:23 AM, Ignacio Diaz-Emparanza wrote:
> Half the world uses the decimal comma.
> This strict CSV format cannot be an international standard. It is not
> "internationally feasible" because of the ambiguity of the comma.
>
This is simply a non-sequitur.
We currently have an unfortunate situation:
some countries have not yet transitioned to a period
for the decimal mark. This means casual communication
is plagued by an ambiguity.
This need not be the case for scientific communication.
There is a standard. Use it. It is really that simple.
Well ... almost: we need some way to communicate adherence to the standard.
That is the role that should be but currently is not played by the .csv
extension.
Note that this lack of ambiguity is already present in almost
all computing languages: the dot is the decimal separator.
Everyone needs to bite the bullet and accept that this is
**already** an international standard, with unfortunately poor adherence.
Science is inherently international and needs single standards.
Arguing for the use of the comma is like arguing for the use of the inch.
Cheers,
Alan
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
--
Clive Nicholas
"My colleagues in the social sciences talk a great deal about methodology.
I prefer to call it style." -- Freeman J. Dyson
4:46 p.m.
Am 02.02.2016 um 22:19 schrieb Alan G Isaac:
Once again, a problem for data analysis arises when
someone insists that at **COMMA** separated values
file should be allowed to use commas for some other purpose.
Stick to the RFC and these problems go away:
https://tools.ietf.org/html/rfc4180
Interestingly, the text at this link contains this:
'Implementors should "be conservative in what you do, be liberal in what
you accept from others" (RFC 793 [8]) when processing CSV files."'
I think this is what gretl is doing.
But there is some truth to what you're saying, especially mentioning
programming languages. That's why gretl's language hansl has always kept
the decimal point as mandatory (despite some controversy there as well).
cheers,
sven
5:02 p.m.
On Fri, 5 Feb 2016, Sven Schreiber wrote:
But there is some truth to what you're saying, especially
mentioning
programming languages. That's why gretl's language hansl has always kept
the decimal point as mandatory (despite some controversy there as well).
Hem hem.
IIRC, it took quite a bit of discussion, and at times an overly
confrontational attitude on my side, to take this result home.
I'm trying to keep as quiet as possible in this discussion. I believe that
everybody who's been around the gretl community for some time knows what
my opinion is and I'll just refrain from voicing it again, because I
appreciate the value of a compromise that took so much work (and
patience) to reach.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
1:58 p.m.
Thank you for the fix, it works great. Thank you for compromising... :-)
I think gretl is a really great package. What distinguishes gretl from the other packages
is its user friendliness. And it's a feature like this easy importing that makes gretl
stand out. If we want "normal people" to start using math and statistics in real
life, we need to help them. They have enough to do to convince their math-hating
colleagues and bosses about these tools in the first place. So I think it's only fair
that they make up a bit for the lost time by doing a quick data import...
Kind regards,
Thorsten
-----Ursprüngliche Nachricht-----
Von: gretl-users-bounces(a)lists.wfu.edu [mailto:gretl-users-bounces@lists.wfu.edu] Im
Auftrag von Allin Cottrell
Gesendet: Dienstag, 2. Februar 2016 21:50
An: Gretl list
Betreff: Re: [Gretl-users] Input of csv data - does the German version work as intended?
On Tue, 2 Feb 2016, Allin Cottrell wrote:
<csv>
date;Erster;Hoch;Tief;Schlusskurs;Stuecke;Volumen
23.02.2015;71,16;71,24;70,6;70,91;172.975;12.251.361
20.02.2015;70,55;70,92;70,22;70,62;752.159;53.136.228
19.02.2015;70,74;71,06;69,93;70,73;576.821;40.735.744
...
</csv>
Gretl running in English gets everything right automatically, but the
importation fails when running in German -- in fact, in any locale
that uses ',' as decimal character and with "Use locale setting for
decimal point" selected in gretl's Preferences.
This is now fixed in git and snapshots.
When gretl realized that '.' must be a thousands marker, she revised her opinion
about ',', now taking it to be the decimal separator. The consequences of this
switch were correctly handled for the case where the local decimal separator is dot, but
not otherwise. Now they're handled in both cases.
Allin
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
10:49 a.m.
Am 02.02.2016 um 21:50 schrieb Allin Cottrell:
On Tue, 2 Feb 2016, Allin Cottrell wrote:
> <csv>
> date;Erster;Hoch;Tief;Schlusskurs;Stuecke;Volumen
> 23.02.2015;71,16;71,24;70,6;70,91;172.975;12.251.361
> 20.02.2015;70,55;70,92;70,22;70,62;752.159;53.136.228
> 19.02.2015;70,74;71,06;69,93;70,73;576.821;40.735.744
> ...
> </csv>
>
> Gretl running in English gets everything right automatically, but the
> importation fails when running in German -- in fact, in any locale
> that uses ',' as decimal character and with "Use locale setting for
> decimal point" selected in gretl's Preferences.
This is now fixed in git and snapshots.
Thanks!
The file did look confusing to my (German) eyes at first, because of the
thousands separators ("."). But it must be acknowledged that it is a
perfectly structured delimited text file in a German context:
- The date format with the dot "DD.MM.YYYY" is absolutely the standard
way in Germany.
- The comma is used according to the German locale.
- The dot as well (although not quite as common of course).
- And using the semicolon to delimit stuff is what I prefer as well,
given the ambiguity of the comma. (Little anekdote: I used the semicolon
as an argument separator in a formula in a paper, and one referee
complained that it was unusual and I should use a comma. Struck me as a
slightly narrow-minded comment.).
- The fact that the dates are "reversed" is only true from the point of
view of gretl's and others' conventions. Having stuff ordered with
newest first is also very common.
So I'm glad gretl supports this structure.
cheers,
sven
4:04 a.m.
Am 01.02.2016 um 15:20 schrieb Wingenroth, Thorsten:
Hi,
Gretl let me down L Full of enthusiasm I praised Gretl to my students,
but they couldn’t even import my csv files with their current release!
Only some general comments: I, too, have experienced problems getting
some csv files to be imported, correctly. Mostly it has had to do with
the specification of dates, so I often have to re-tell gretl the
structure of the dataset.
However, I also very often have (other) problems when I try to open csv
files in Excel. Most of the time I have to pre-edit the csv file in a
clever text editor to get it right.
So my conclusion over the years has been that pre-adjusting the csv
files is almost unavoidable.
But don't get me wrong, gretl should do everything it can to get it right.
cheers,
sven
3094
days inactive
3099
days old
15 comments
7 participants
participants (7)
-
Alan G Isaac -
Allin Cottrell -
Clive Nicholas -
Ignacio Diaz-Emparanza -
Riccardo (Jack) Lucchetti -
Sven Schreiber -
Wingenroth, Thorsten