12:55 p.m.
On Thu, January 4, 2007 13:16, Sven Schreiber wrote:
> First, let's define the concept of a "binary
bundle". A binary bundle is a
> binary file which contains one or more objects. An object can be
>
> 1) a scalar
> 2) a matrix
> 3) a dataset
> 4) a model
>
I would also add: 5) some text, comment, etc.
Not difficult, ok.
Why does it have to be binary? Human-readable is an advantag IMO.
Unless
you have tons of data, but I doubt you would be using gretl then (no
offense...).
Point taken. If I knew more than I do about this, the best thing would
actually be xml. Trouble is, I know how to write functions for reading/writing
binary stuff, with xml I'm lost. But I could learn.
About data size: no offense taken, sure, but I'd like you to try what happens
with big datasets. In my experience, gretl is actually very efficient. I've
handled datasets with hundreds of thousands of observations much better than
in other packages.
> I think this would provide a very neat way to do what I assume
you need.
> However, the coding effort is definitely not trivial. Could we schedule this
> for 1.7.0?
What if you start out just with the ability to read in "bundles" with
genr/matrix/print commands (for scalars/matrices/text). Essentially just
opening the file and executing the literal contents as if the user typed
them in directly. like a limited version of 'run'. That wouldn't be so
difficult, would it?
Mmmhhh... I don't like this very much. Setting up a specialised version of
"run" that we know in advance is bound to be scrapped soon looks like a waste
of resources to me...
(Should we take this off the user list? but keep me cc-ed please, or
maybe I should subscribe to gretl-devel...)
Indeed. I've cc-ed this message to gretl-devel.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Facoltà di Economia "G. Fuà"
Ancona
1:41 p.m.
New subject: [Gretl-users] error with null list
[how do I subscribe to gretl-devel?]
Riccardo (Jack) Lucchetti schrieb:
About data size: no offense taken, sure, but I'd like you to try what happens
with big datasets. In my experience, gretl is actually very efficient. I've
handled datasets with hundreds of thousands of observations much better than
in other packages.
Right, but I guess you wouldn't want to copy huge datasets back and
forth between different engines, binary or not. So then you would
probably use the external engine exclusively w/o a gretl interface.
> What if you start out just with the ability to read in
"bundles" with
> genr/matrix/print commands (for scalars/matrices/text). Essentially just
> opening the file and executing the literal contents as if the user typed
> them in directly. like a limited version of 'run'. That wouldn't be so
> difficult, would it?
Mmmhhh... I don't like this very much. Setting up a specialised version of
"run" that we know in advance is bound to be scrapped soon looks like a waste
of resources to me...
Is it really going to be that soon? :-) But seriously, in my naive mind
that new 'read_bundle' command would initially be like a limited run,
with one big exception: it could be used in functions. But afterwards it
would be extended to be able to do what you described (including models
etc.). Why do you see extending the functionality of 'read_bundle' in
the future as scrapping it?
All I'm saying is that it would be nice if gretl's functions could not
just pass information to the external world (store, outfile), but also
receive/read information.
Think for example of a function package that is written exclusively in
gretl script, no external engine, BUT which has very many (tens,
hundreds?) of parameters. Instead of manually specifying all parameters
every time you run the function, it would be nice to save a parameter
configuration and let the function itself load it. But since currently a
function cannot read any files, that's impossible.
So I think there are use cases for reading files within functions beyond
the one I'm currently interested in.
-sven
2:16 p.m.
New subject: [Gretl-users] error with null list
On Thu, January 4, 2007 14:41, Sven Schreiber wrote:
[how do I subscribe to gretl-devel?]
http://ricardo.ecn.wfu.edu/mailman/listinfo/gretl-devel
> About data size: no offense taken, sure, but I'd like you to
try what
> happens
> with big datasets. In my experience, gretl is actually very efficient. I've
> handled datasets with hundreds of thousands of observations much better than
> in other packages.
Right, but I guess you wouldn't want to copy huge datasets back and
forth between different engines, binary or not. So then you would
probably use the external engine exclusively w/o a gretl interface.
Yes, with an emphasis on "probably", with all the implications.
So I think there are use cases for reading files within functions
beyond
the one I'm currently interested in.
You know what? I'm starting to think that the whole issue could by nicely
bypassed if matrices could be stored in a gdt file. Then, all you'd need on
the gretl side would be an "append" command. Granted, this shifts some of the
burden on the external app, as it would need to create gdt files.
Allin? Have you woken up? :-)
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Facoltà di Economia "G. Fuà"
Ancona
2:29 p.m.
New subject: [Gretl-users] error with null list
Riccardo (Jack) Lucchetti schrieb:
You know what? I'm starting to think that the whole issue could by nicely
bypassed if matrices could be stored in a gdt file. Then, all you'd need on
the gretl side would be an "append" command. Granted, this shifts some of the
burden on the external app, as it would need to create gdt files.
Perfectly fine with me... I don't want to let you do my work -- well,
not more than what's necessary ;-)
-sven
3:39 p.m.
New subject: [Gretl-users] error with null list
On Thu, 4 Jan 2007, Riccardo (Jack) Lucchetti wrote:
Allin? Have you woken up? :-)
Yes, just trying to catch up! It seems to me this morning that
implementing "run" inside functions is not as difficult as I
thought -- and if so, this would be a nice quick way of enabling
what Sven wants. Other ideas, such as enabling the storage of
matrices in .gdt files, are worth considering too.
Allin.
4:27 p.m.
New subject: [Gretl-users] error with null list
On Thu, January 4, 2007 16:39, Allin Cottrell wrote:
On Thu, 4 Jan 2007, Riccardo (Jack) Lucchetti wrote:
> Allin? Have you woken up? :-)
Yes, just trying to catch up! It seems to me this morning that
implementing "run" inside functions is not as difficult as I
thought -- and if so, this would be a nice quick way of enabling
what Sven wants. Other ideas, such as enabling the storage of
matrices in .gdt files, are worth considering too.
Ok. If you don't see "include/run" in function as problematic, so be it. But
I
still think that for this sort of things we need a better long-term solution.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Facoltà di Economia "G. Fuà"
Ancona
4:16 a.m.
New subject: [Gretl-users] error with null list
On Thu, 4 Jan 2007, Riccardo (Jack) Lucchetti wrote:
Ok. If you don't see "include/run" in function as
problematic,
so be it. But I still think that for this sort of things we need
a better long-term solution.
I do see "include" as problematic (since that's designed for
importing function definitions), but "run" is not so bad. In
current CVS you can invoke "run" inside functions (though this is
not much tested).
As for the bundles, there's now a fairly full mechanism in place
for "serializing" gretl objects (models, matrices, what have you)
in XML, and for reading these back from XML. This is used in the
new-style "session" files. It could be used more widely. One
issue, though, is how clever 3rd party programs would be at
reading the XML. (There are DTDs available, if that's any help).
Allin.
10:02 a.m.
New subject: [Gretl-users] error with null list
[I'm now subscribed to gretl-devel]
Allin Cottrell schrieb:
I do see "include" as problematic (since that's designed for importing
function definitions), but "run" is not so bad. In current CVS you can
invoke "run" inside functions (though this is not much tested).
Wow, that's great, I will test!
As for the bundles, there's now a fairly full mechanism in place for
"serializing" gretl objects (models, matrices, what have you) in XML,
and for reading these back from XML. This is used in the new-style
"session" files. It could be used more widely. One issue, though, is
how clever 3rd party programs would be at reading the XML. (There are
DTDs available, if that's any help).
Does "new-style" mean from current cvs, or since 1.6.0?
And could you elaborate on how to "read these back from XML"? Is there
any script interface for that? Can it be appended to an already opened
workfile, as Jack suggested?
But the latter questions are more of long-term interest, if 'run' works
as expected everything will be fine for now.
I really appreciate the quick action, and I will try not to request too
many new features in the future...
cheers,
sven
5:35 p.m.
New subject: [Gretl-users] error with null list
Sven Schreiber schrieb:
[I'm now subscribed to gretl-devel]
Allin Cottrell schrieb:
> I do see "include" as problematic (since that's designed for importing
> function definitions), but "run" is not so bad. In current CVS you can
> invoke "run" inside functions (though this is not much tested).
Wow, that's great, I will test!
This works like a charm for me! Tested with 'matrix' and 'print'
commands in the file which is run. Very cool, now everything fits together!
Of course this immediately leads to new questions: Is it possible to
return more than one matrix from the function? I tried to adapt section
10.2 of the user guide, but only got error messages.
(e.g. for "return matrix b1, matrix b2" I got the message 'variable name
contains invalid characters')
thanks and have a good weekend,
sven
6:01 p.m.
New subject: [Gretl-users] error with null list
On Fri, 5 Jan 2007, Sven Schreiber wrote:
>> I do see "include" as problematic (since that's
designed for importing
>> function definitions), but "run" is not so bad. In current CVS you
can
>> invoke "run" inside functions (though this is not much tested).
This works like a charm for me! Tested with 'matrix' and 'print'
commands in the file which is run. Very cool, now everything
fits together!
Glad to hear it!
Of course this immediately leads to new questions: Is it
possible to return more than one matrix from the function? I
tried to adapt section 10.2 of the user guide, but only got
error messages.
You can do this, but the "extra" returns have to be indirect. You
need to update your copy of the user guide!
http://ricardo.ecn.wfu.edu/pub/gretl/manual/PDF/
(Though there should be a way of doing this via the GUI.)
Trivial example:
function matfun (matrix *B, matrix *C)
matrix A = mnormal(2,2)
matrix B = mnormal(2,2)
matrix C = mnormal(2,2)
return matrix A
end function
nulldata 10
matrix B
matrix C
matrix A = matfun(&B, &C)
print A B C
Allin.
8:46 p.m.
New subject: [Gretl-users] error with null list
Allin Cottrell schrieb:
On Fri, 5 Jan 2007, Sven Schreiber wrote:
You can do this, but the "extra" returns have to be indirect. You need
to update your copy of the user guide!
Thanks for the hint, the method reminds me alot of Ox. I always thought
there isn't an update available because on the title page it still says
"July 2006".
-sven
5:12 p.m.
New subject: [Gretl-users] error with null list
On Fri, 5 Jan 2007, Sven Schreiber quoted me:
> As for the bundles, there's now a fairly full mechanism in
> place for "serializing" gretl objects (models, matrices, what
> have you) in XML, and for reading these back from XML. This is
> used in the new-style "session" files. It could be used more
> widely. One issue, though, is how clever 3rd party programs
> would be at reading the XML. (There are DTDs available, if
> that's any help)
and asked:
Does "new-style" mean from current cvs, or since 1.6.0?
The "new-style" session management was introduced in 1.6.0, but it
was a bit buggy. Several issues have been fixed in CVS.
And could you elaborate on how to "read these back from
XML"? Is
there any script interface for that? Can it be appended to an
already opened workfile, as Jack suggested?
There's no script interface at present. But lib/src/gretl_xml.c
contains a set of simple C functions (based on libxml2) for
writing various sorts of data to XML and reading it back, for
example gretl_xml_put_matrix() and gretl_xml_get_matrix().
Allin.
6558
days inactive
6562
days old
11 comments
3 participants
participants (3)
-
Allin Cottrell -
Riccardo (Jack) Lucchetti -
Sven Schreiber