Re: [Gretl-devel] opening large spss file kills the app without error message

Wow!!!!!!!!!!!!!! That's great!!!!!!!!!!! Will try with the current snapshot! For the moment neigher R (haven) nor python (pandas) can't open this file. Best, Boris Demeshev пн, 18 мар. 2019 г. в 23:19, Allin Cottrell <cottrell(a)wfu.edu&gt;: > > On Sat, 16 Mar 2019, Boris Demeshev wrote: > > > Thanks a lot for the support!!! > > > > There are 4 files (individuals spss, individuals stata, households > > spss, households stata). For the moment I can not open Individuals > > files (neither stata nor spss). Will check with current build and > > again next week! :) > > I've been working with the Stata individuals (longitudinal) file, > USER_RLMS-HSE_IND_1994_2017_v2_eng.dta > > There was a bug (a typo, really) in our dta reader which prevented > it from reading the "map" of offsets properly when some elements > were bigger than 32-bit integers. That's now fixed in git and > snapshots and I can open the file OK on a machine with 16 GB of RAM. > > It's a struggle on an older machine with 8 GB. The uncompressed > Stata file is about 3.8 GB, and the gretl in-memory representation > of the full dataset takes up almost 7 GB. That's because all data > are stored as doubles in gretl -- we don't have a "byte" data type > as in Stata, which permits economical encoding of dummy variables. > > However, if you plan to use only a relatively small subset of the > 2723 series, you can open the dta file on a machine with plenty of > RAM and save a subset in gretl native format. The reduced file will > then be readable on most machines. > > Allin > _______________________________________________ > Gretl-devel mailing list > Gretl-devel(a)lists.wfu.edu > http://lists.wfu.edu/mailman/listinfo/gretl-devel