[Gretl-devel] Re: problem with large gdtb panel datafile and index variables

On Fri, Jul 14, 2023 at 8:59 AM Sven Schreiber <sven.schreiber(a)fu-berlin.de&gt; wrote: > I'm seeing a problem handling a fairly large panel dataset in the GUI. > It started with a ~200MB binary gdtb file that was a recognized panel, > with index variables "iso" (time, iso dates) and "STATIONS_ID". > > This had lots of unneeded obs with mostly missings, so I saved a reduced > version of the dataset (after "smpl --no-all-missing <somelist>"), > yielding a file with about 100MB. As expected (?), the official panel > structure was lost there, so I wanted to re-instate it, using the nice > GUI dataset structure tool. Basically, this ended up in a non-responsive > gretl which had to be killed by the OS. (I waited several minutes before > doing that.) The GUI steps for doing this are: 1. Select "panel" as the target structure. 2. Select "Use index variables" as the organization. 3. Select Unit and Time variables from the candidates offered. 4. Specify the panel time dimension. 5. Confirm the selections. I tried mocking up a fairly hefty panel (though no doubt a good deal smaller than yours) and I got a pause of several seconds between steps 3 and 4. Is that the point at which gretl became unresponsive in your case?

There's a potentially expensive check going on at that stage; I'm not sure it's really needed but if so it can probably be made more efficient.