[Gretl-devel] Re: invalid compressed data
On Tue, 1 Oct 2019, Artur Tarassow wrote:
I am currently working with a 2.8GB csv data file which I stored
as a gdtb file (compressed to about 170MB). However, trying to
open the gdtb file yields an error (see attached screenshot) with
the message that the data is not compressed in a valid manner.
Smaller gdtb files work fine though.
There are two code paths for zipping/unzipping in gretl (other than
just straight gzip compression or extraction of a single file, which
is handled by the zlib API).
1) The gretlzip plugin module, which is based on the code for the
Info-ZIP executable "zip" (Mark Adler et al), version 2.31. This is
now somewhat dated.
2) Libgsf-1 (gsf = gnome structured file, used as back-end for
gnumeric).
I suspect that 1) may be subject to a 4GB limit but 2) is not.
In years past we considered use of libgsf as experimental, and you
had to specifically request it in a gretl build by passing
--with-gsf to the configure script. I guess it's time to change
that. Now (current git) if libgsf >= 1.14.31 is found when
configuring, it will be used by preference. (Of course, you'd need
the "dev" or "devel" libgsf package.)
Allin