[Gretl-devel] Re: problem with large gdtb panel datafile and index variables
On Fri, Jul 14, 2023 at 8:59 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
I'm seeing a problem handling a fairly large panel dataset in the GUI.
It started with a ~200MB binary gdtb file that was a recognized panel,
with index variables "iso" (time, iso dates) and "STATIONS_ID".
This had lots of unneeded obs with mostly missings, so I saved a reduced
version of the dataset (after "smpl --no-all-missing <somelist>"),
yielding a file with about 100MB. As expected (?), the official panel
structure was lost there, so I wanted to re-instate it, using the nice
GUI dataset structure tool. Basically, this ended up in a non-responsive
gretl which had to be killed by the OS. (I waited several minutes before
doing that.)
The GUI steps for doing this are:
1. Select "panel" as the target structure.
2. Select "Use index variables" as the organization.
3. Select Unit and Time variables from the candidates offered.
4. Specify the panel time dimension.
5. Confirm the selections.
I tried mocking up a fairly hefty panel (though no doubt a good deal
smaller than yours) and I got a pause of several seconds between steps
3 and 4. Is that the point at which gretl became unresponsive in your
case? There's a potentially expensive check going on at that stage;
I'm not sure it's really needed but if so it can probably be made more
efficient.
Allin