3:56 p.m.
Hi,
I think I'm experiencing a small glitch with gretl's handling of
cross-platform line endings on Windows when printing to a file.
Consider this:
<hansl>
s = readfile("test.txt") # this file is attached
set verbose off
outfile test2.inp # this is also attached
print s
end outfile
</hansl>
The newlines in test.txt are Windows-style CRLF.
In test2.inp each CRLF appears to be replaced by CRCRLF. In gretl's
editor this shows up as a double line break (empty lines).
This is with 2020e.
thanks
sven
6:05 p.m.
On Fri, 11 Dec 2020, Sven Schreiber wrote:
I think I'm experiencing a small glitch with gretl's handling
of
cross-platform line endings on Windows when printing to a file.
Consider this:
<hansl>
s = readfile("test.txt") # this file is attached
set verbose off
outfile test2.inp # this is also attached
print s
end outfile
</hansl>
The newlines in test.txt are Windows-style CRLF.
Heh, actually they're not (in the attachment). The "newlines" are
old-Mac style, CR only. The (hex) bytes are as follows (CR = 0d; LF
= 0a, not present):
68 65 79 0d 68 6f 0d 6c 65 74 27 73 0d 67 6f
(with no newline at the end). If I convert it to CR-LF line endings
(using emacs), then use readfile() on the modified file and print
the result in gretl, I get correct results -- if by "correct" we
mean one word per line with no spaces between lines.
Allin
10:21 p.m.
Am 11.12.2020 um 19:05 schrieb Allin Cottrell:
On Fri, 11 Dec 2020, Sven Schreiber wrote:
> I think I'm experiencing a small glitch with gretl's handling of
> cross-platform line endings on Windows when printing to a file.
>
> Consider this:
>
> <hansl>
>
> s = readfile("test.txt") # this file is attached
> set verbose off
> outfile test2.inp # this is also attached
> print s
> end outfile
>
> </hansl>
>
> The newlines in test.txt are Windows-style CRLF.
Heh, actually they're not (in the attachment).
I see that in the attachment, too - but this must be the result of some
conversion in the email transfer process (maybe already in my email
program, don't know). I'm relatively sure that when I attached the file
it was like I described.
Perhaps somebody else could test on Windows with a self-created file
with CR+LF newlines.
BTW, I also observed that the display/printing of the string within
gretl in the output window was fine. The phenomenon only occurred when
saving to a file.
thanks
sven
1:30 a.m.
On Fri, 11 Dec 2020, Sven Schreiber wrote:
Am 11.12.2020 um 19:05 schrieb Allin Cottrell:
> On Fri, 11 Dec 2020, Sven Schreiber wrote:
>
>> I think I'm experiencing a small glitch with gretl's handling of
>> cross-platform line endings on Windows when printing to a file.
>>
>> Consider this:
>>
>> <hansl>
>>
>> s = readfile("test.txt") # this file is attached
>> set verbose off
>> outfile test2.inp # this is also attached
>> print s
>> end outfile
>>
>> </hansl>
>>
>> The newlines in test.txt are Windows-style CRLF.
>
> Heh, actually they're not (in the attachment).
I see that in the attachment, too - but this must be the result of some
conversion in the email transfer process (maybe already in my email
program, don't know). I'm relatively sure that when I attached the file
it was like I described.
Hmm, I know things can be messed about by email, but it would be
very surprising if a mail transfer agent turned CRLF (well known as
the Windows standard) into CR (definitely a Mac-only thing).
Perhaps somebody else could test on Windows with a self-created file
with CR+LF newlines.
BTW, I also observed that the display/printing of the string within
gretl in the output window was fine. The phenomenon only occurred when
saving to a file.
OK, I'll take a look at that.
Allin
4:51 p.m.
Am 12.12.2020 um 02:30 schrieb Allin Cottrell:
On Fri, 11 Dec 2020, Sven Schreiber wrote:
> BTW, I also observed that the display/printing of the string
within
> gretl in the output window was fine. The phenomenon only occurred when
> saving to a file.
OK, I'll take a look at that.
Here's a slight extension of the previous script, where newtest.txt has
the contents: "no\nsleep\ntill\nBrooklyn", and according to what I see
in the SciTE editor it has CRLF line endings. Counting CRLF as two
characters there are 25 chars in the file (ignoring EOF).
<hansl>
s = readfile("newtest.txt")
eval nelem(s) # gives 25
eval strlen(s) # 25
set verbose off
outfile newtest2.inp
print s
end outfile
s2 = readfile("newtest2.inp")
eval nelem(s2) # 30
eval strlen(s2) # 30
</hansl>
The file got 5 new characters, and closer inspection shows this content:
noCRCRLFsleepCRCRLFtillCRCRLFBrooklynCRLF
This time there's no email involved, just gretl I think!
on snapshot Dec 9th
cheers
sven
9:13 p.m.
On Sat, 12 Dec 2020, Sven Schreiber wrote:
Am 12.12.2020 um 02:30 schrieb Allin Cottrell:
> On Fri, 11 Dec 2020, Sven Schreiber wrote:
>> BTW, I also observed that the display/printing of the string within
>> gretl in the output window was fine. The phenomenon only occurred when
>> saving to a file.
>
> OK, I'll take a look at that.
Here's a slight extension of the previous script, where newtest.txt has
the contents: "no\nsleep\ntill\nBrooklyn", and according to what I see
in the SciTE editor it has CRLF line endings. Counting CRLF as two
characters there are 25 chars in the file (ignoring EOF).
<hansl>
s = readfile("newtest.txt")
eval nelem(s) # gives 25
eval strlen(s) # 25
set verbose off
outfile newtest2.inp
print s
end outfile
s2 = readfile("newtest2.inp")
eval nelem(s2) # 30
eval strlen(s2) # 30
</hansl>
The file got 5 new characters, and closer inspection shows this content:
noCRCRLFsleepCRCRLFtillCRCRLFBrooklynCRLF
This time there's no email involved, just gretl I think!
This whole business is an annoyance and stumbling-block for coders
almost as severe as Microsoft's persistence with an unholy mixture
of 8-bit "system codepage" and UTF-16. They persist in using two
bytes to indicate what everyone else indicates in one byte (start a
new line), because in the stone age this required two operations of
a teletype machine: crank the paper up a line, and return the
print-head to the left.
Getting this "right" in a cross-platform program is a horror-show.
That said, it might now be better in current git and snapshot.
Keyword: "binary mode" or "b" when calling the C library function
fopen() for reading or writing. It does nothing except when you're
handling text files on Windows. But when should you use it, and when
not?
Allin
1:22 p.m.
Am 12.12.2020 um 22:13 schrieb Allin Cottrell:
On Sat, 12 Dec 2020, Sven Schreiber wrote:
>
> The file got 5 new characters, and closer inspection shows this content:
> noCRCRLFsleepCRCRLFtillCRCRLFBrooklynCRLF
>
> This time there's no email involved, just gretl I think!
This whole business is an annoyance and stumbling-block for coders
almost as severe as Microsoft's persistence with an unholy mixture of
8-bit "system codepage" and UTF-16. They persist in using two bytes to
indicate what everyone else indicates in one byte (start a new line),
because in the stone age this required two operations of a teletype
machine: crank the paper up a line, and return the print-head to the left.
Getting this "right" in a cross-platform program is a horror-show. That
said, it might now be better in current git and snapshot.
Keyword: "binary mode" or "b" when calling the C library function
fopen() for reading or writing. It does nothing except when you're
handling text files on Windows. But when should you use it, and when not?
I feel your pain, Allin! Indeed I remember issues like that also in
Python where it was never quite clear to me what the file write mode was
supposed to mean in detail...
Thanks for the fix,
Sven
1475
days inactive
1477
days old
6 comments
2 participants
participants (2)
-
Allin Cottrell -
Sven Schreiber