[Gretl-devel] Re: invalid compressed data

On Tue, 1 Oct 2019, Artur Tarassow wrote: > I am currently working with a 2.8GB csv data file which I stored as a > gdtb file (compressed to about 170MB). However, trying to open the > gdtb file yields an error (see attached screenshot) with the message > that the data is not compressed in a valid manner. Smaller gdtb files > work fine though. There are two code paths for zipping/unzipping in gretl (other than just straight gzip compression or extraction of a single file, which is handled by the zlib API). 1) The gretlzip plugin module, which is based on the code for the Info-ZIP executable "zip" (Mark Adler et al), version 2.31. This is now somewhat dated. 2) Libgsf-1 (gsf = gnome structured file, used as back-end for gnumeric). I suspect that 1) may be subject to a 4GB limit but 2) is not. In years past we considered use of libgsf as experimental, and you had to specifically request it in a gretl build by passing --with-gsf to the configure script. I guess it's time to change that. Now (current git) if libgsf >= 1.14.31 is found when configuring, it will be used by preference. (Of course, you'd need the "dev" or "devel" libgsf package.)