1:32 p.m.
Hi,
(with gretl snapshot Apr 4th) I tried to export the penngrow.gdt data
file to Stata format (dta). Opening it in Stata 14 works fine at first,
but the values are totally rubbish.
Going via csv works OK.
thanks
sven
1:54 p.m.
On Tue, 9 Apr 2019, Sven Schreiber wrote:
Hi,
(with gretl snapshot Apr 4th) I tried to export the penngrow.gdt data
file to Stata format (dta). Opening it in Stata 14 works fine at first,
but the values are totally rubbish.
Going via csv works OK.
It works ok with Stata 13 too.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:59 p.m.
On Tue, 9 Apr 2019, Sven Schreiber wrote:
Hi,
(with gretl snapshot Apr 4th) I tried to export the penngrow.gdt data
file to Stata format (dta). Opening it in Stata 14 works fine at first,
but the values are totally rubbish.
Going via csv works OK.
Here the gretl-written penngrow.dta opens OK and has correct data
values in Stata 12. (Also the round-trip of opening the dta with gretl
works fine.)
Could you maybe send me your penngrow.dta offlist so we can check if
the problem is on the writing side or the reading side?
Allin
6:26 a.m.
Am 09.04.2019 um 17:59 schrieb Allin Cottrell:
On Tue, 9 Apr 2019, Sven Schreiber wrote:
Here the gretl-written penngrow.dta opens OK and has correct data
values
in Stata 12. (Also the round-trip of opening the dta with gretl works
fine.)
Could you maybe send me your penngrow.dta offlist so we can check if the
problem is on the writing side or the reading side?
Will do that, but was anybody able to test with Stata 14 (or 15)?
thanks
sven
8:03 a.m.
Am 10.04.2019 um 08:26 schrieb Sven Schreiber:
Am 09.04.2019 um 17:59 schrieb Allin Cottrell:
> On Tue, 9 Apr 2019, Sven Schreiber wrote:
> Here the gretl-written penngrow.dta opens OK and has correct data values
> in Stata 12. (Also the round-trip of opening the dta with gretl works
> fine.)
>
> Could you maybe send me your penngrow.dta offlist so we can check if the
> problem is on the writing side or the reading side?
Will do that, but was anybody able to test with Stata 14 (or 15)?
Well, the file is small and is public data anyway, so I'm attaching it here.
In the meantime I tried reading it with pandas (Python). It failed at
first with the error "unpack requires a bytes object of length 4". I had
to set convert_categoricals=False, then it worked.
The dta file was produced on Win10 with Stata MP 14.2, both are 64bit
versions AFAIK.
thanks
sven
12:23 p.m.
On Wed, 10 Apr 2019, Sven Schreiber wrote:
Well, the file is small and is public data anyway, so I'm
attaching it here.
In the meantime I tried reading it with pandas (Python). It failed at
first with the error "unpack requires a bytes object of length 4". I had
to set convert_categoricals=False, then it worked.
Thanks, that should give us a clue, eventually!
The dta file was produced on Win10 with Stata MP 14.2, both are
64bit
versions AFAIK.
The dta file in your attachment (written by gretl) opens in gretl OK
but the data are corrupted from observation 51 onward. Same thing with
Stata 12.
Just to get clear: when the "original" dta file written by Stata 14
was read into gretl, were the data OK at that point?
Could you maybe send also the original dta?
Allin
12:48 p.m.
Am 10.04.2019 um 14:23 schrieb Allin Cottrell:
On Wed, 10 Apr 2019, Sven Schreiber wrote:
> The dta file was produced on Win10 with Stata MP 14.2, both are 64bit
> versions AFAIK.
I'm sorry, I meant it was _imported_ into 64bit Stata.
The dta file in your attachment (written by gretl) opens in gretl OK
but the data are corrupted from observation 51 onward. Same thing with
Stata 12.
Just to get clear: when the "original" dta file written by Stata 14
was read into gretl, were the data OK at that point?
Again sorry for the confusion. There is no original dta. The original
file is the penngrow.gdt file shipped with gretl. Then exported by gretl
to dta, from which on the problems ensued.
thanks
sven
7:09 p.m.
On Wed, 10 Apr 2019, Sven Schreiber wrote:
Am 10.04.2019 um 14:23 schrieb Allin Cottrell:
> On Wed, 10 Apr 2019, Sven Schreiber wrote:
>
>> The dta file was produced on Win10 with Stata MP 14.2, both are 64bit
>> versions AFAIK.
I'm sorry, I meant it was _imported_ into 64bit Stata.
OK. The problem seems to be independent of the Stata version, and
amounts to the fact that gretl's stata exporter is broken on Windows.
If I produce a dta of the penngrow data on Linux, it's fine; if I do
so on Windows it's broken from observation 50 onward.
I can see that the hex dumps of the dta files produced on Linux and
Windows diverge at byte 00006ec, in the course of writing a row of
data. So far I have no idea why that's happening -- the copious
debugging statements that I've added have not yet provided a clue.
I'll take another look tomorrow.
Allin
5:40 p.m.
On Wed, 10 Apr 2019, Allin Cottrell wrote:
[Re. Sven's discovery that a Stata .dta file written by gretl on
Windows, via "/File/Export data", was corrupted]
OK. The problem seems to be independent of the Stata version, and
amounts to the fact that gretl's stata exporter is broken on
Windows. If I produce a dta of the penngrow data on Linux, it's
fine; if I do so on Windows it's broken from observation 50 onward.
This is now fixed in git and snapshots. We were using the POSIX
open/write API for producing the binary dta file. I thought this would
be OK on Windows via mingw but apparently it is not. We've now
switched to use of the standard C fopen/fwrite API.
Allin
8:38 a.m.
Am 11.04.2019 um 19:40 schrieb Allin Cottrell:
On Wed, 10 Apr 2019, Allin Cottrell wrote:
[Re. Sven's discovery that a Stata .dta file written by gretl on
Windows, via "/File/Export data", was corrupted]
> OK. The problem seems to be independent of the Stata version, and
> amounts to the fact that gretl's stata exporter is broken on Windows.
> If I produce a dta of the penngrow data on Linux, it's fine; if I do
> so on Windows it's broken from observation 50 onward.
This is now fixed in git and snapshots. We were using the POSIX
open/write API for producing the binary dta file. I thought this would
be OK on Windows via mingw but apparently it is not. We've now switched
to use of the standard C fopen/fwrite API.
Thanks Allin. I have some anecdotal evidence that it works better now.
(@Artur:) Since Stata is important for many people or for collaboration
perhaps this would be worth a tweet ?
cheers
sven
7:01 a.m.
Am 07.05.19 um 10:38 schrieb Sven Schreiber:
Am 11.04.2019 um 19:40 schrieb Allin Cottrell:
> On Wed, 10 Apr 2019, Allin Cottrell wrote:
>
> [Re. Sven's discovery that a Stata .dta file written by gretl on
> Windows, via "/File/Export data", was corrupted]
>
>> OK. The problem seems to be independent of the Stata version, and
>> amounts to the fact that gretl's stata exporter is broken on Windows.
>> If I produce a dta of the penngrow data on Linux, it's fine; if I do
>> so on Windows it's broken from observation 50 onward.
>
> This is now fixed in git and snapshots. We were using the POSIX
> open/write API for producing the binary dta file. I thought this would
> be OK on Windows via mingw but apparently it is not. We've now switched
> to use of the standard C fopen/fwrite API.
Thanks Allin. I have some anecdotal evidence that it works better now.
(@Artur:) Since Stata is important for many people or for collaboration
perhaps this would be worth a tweet ?
Good idea, I've just sent a tweet.
Best,
Artur
2036
days inactive
2068
days old
10 comments
4 participants
participants (4)
-
Allin Cottrell -
Artur Tarassow -
Riccardo (Jack) Lucchetti -
Sven Schreiber