Am 01.10.19 um 22:22 schrieb Sven Schreiber:
Am 01.10.2019 um 22:05 schrieb Artur Tarassow:
> Sorry, for cross-posting. The message I received this morning I just
> attached again.
> So it seems this issue is related to zip as a format which has a 4gb
> limit. Is there a way to circumvent this - for instance by switching
> to gzip or so (just in case the user can control this).
Hi Artur, I believe the 4GB limit is inherent in the zip format, not a
limitation of the tool(s). The successor format seems to be zip64, maybe
gretl should switch to using that.
it seems that gretl uses PKZIP. That's what I found in the manual and
also some hint on Allins homepage:
I could not find anything on whether newer versions of PKZIP supports a
file type like zip64.
In the meantime, maybe the file can be split up?
Yes, would be a strategy. I could then join the n-1 splited data sets.
Originally I had the following strategy in mind:
1) Reduce the full data set to a single variable which I use as a filter
to select only some cross-sectional units (around 58,000 out of 110,000).
2) Re-join about 10 series to the remaining 58,000 cross-sectional units
(T=639). However, joining a _single_ series (--ikey has 2 identifying
variables) already takes almost 1h -- so this is no solution for me ;-)
RAM is no issue but loading and storing big CSVs takes quite long. So
gdtb is the recommended alternative.