[Gretl-devel] memory with many discrete variables in large (panel) dataset

Hi, I'd like to raise an issue which is probably quite fundamental in terms of data handling. I'm currently working on a large panel dataset, meaning that gretl occupies more than 600MB of memory with the data loaded. In terms of file sizes, the Stata file version occupies 42MB, the gretl workfile only about 3.5MB. This shows that gretl stores the data very efficiently (by zipping), but OTOH opening and saving takes quite some time. Actually it is much faster even in gretl to import the Stata file instead of the native gretl file. Now I can admit beforehand that I do not actually need most of the variables in there and I can work with a stripped-down version. (However, as a side remark: this is complicated by the fact that I could not delete the unwanted variables in a loop, and when deleting manually I often got the error "too many items selected".) Nevertheless, given the fact that many of the variables in there are discrete integer-valued which could be saved as a signed single byte, and that (I think) I know that gretl stores every variable as double-precision 8-byte numbers, there are huge potential memory savings and speed improvements. Maybe that's something to think about for the longer term. Or are there alternatives that I have missed? thanks, sven