Re: [Gretl-devel] memory with many discrete variables in large (panel) dataset
On Wed, 8 Jan 2014, Sven Schreiber wrote:
BTW, I have always wondered how gretl actually determines the
true/false
value of something like "indicator == 1" given that the series are
stored as floating-point values and the corresponding representation
problems of integers...
Fortunately, dummy variables are not a problem, since both real numbers
0.0 and 1.0 have and exact floating-point representation.
> 2) Writing binary data
>
> Tweaks to our writing of data in text form are be useful, but
> there's no question that if you want raw speed you're better using
> C's fwrite and fread to zap big swathes of bytes from RAM to disk or
> vice versa. I've implemented a --binary option to "store" that
> causes gretl to write out an XML .gdt file containing the metadata
> plus a binary .bdt file containing doubles.
Hm, that sounds as if now a .gdt file could indicate either a
traditional standalone file or a new metadata file, which are quite
different things, no? A new suffix would seem in order. (.mgdt?)
I wouldn't mind something like that, especially because gdt files have
always been self-contained. With the new "binary" thing, instead, the
dataset is spread between 2 files, with the gdt file containing the
metadata only. The problem is, it would be difficult to tell a
'self-contained' gdt file from a 'metadata-only' gdt file from the
outside, which could be a problem in some cases.
Yes I'm a big fan of compression level 1 -- why not make that the
default?
Good idea.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------