This is now working OK on Linux and Windows (and probably OS X), but is not yet documented. You can stick Octave script into a gretl "foreign" block with "language=octave", and can exchange matrices (as with Ox) via the pre-defined Octave functions gretl_loadmat(name1) gretl_export(matrix, name2) The first of these matches up with mwrite(matrix, name1) in gretl, and the second pairs with matrix = mread(name2) in gretl. Now we need a nice example of the use of Octave to illustrate the yet-to-be-written entry for the User's Guide. Any suggestions?

On Wed, 24 Feb 2010, Sven Schreiber wrote: > Allin Cottrell schrieb:

>> >> Now we need a nice example of the use of Octave to illustrate the >> yet-to-be-written entry for the User's Guide. Any suggestions? > Well as Jack mentioned, maybe (cross-) spectral stuff would be a good > area because of complex numbers. For example the "cohesion" measure by > Croux/Forni/Reichlin? Thanks for the suggestion. I was kinda hoping that somebody who knows Octave better than I might actually contribute an example for the manual. The only things that I know how to do in Octave are boring stuff that you could do just as well in gretl.

> And slightly OT: As a pythonista I would of course be interested in > adding numerical Python (NumPy) to the list of supported foreign > languages. I'm ready to help with the import/export functions. What > exactly would be needed? Basically, two functions in the target language, one to import gretl matrices as written by mwrite(), and one to export matrices in the format readable by mread(). Plus one auxiliary function to return the gretl "dotdir". These get written out by gretl and somehow (depending on the language) get "sourced" at startup of "foreign" execution.

The idea of using GPUs for fast number-crunching sounds cool.

Allin Cottrell schrieb: > On Wed, 24 Feb 2010, Sven Schreiber wrote: > >> Allin Cottrell schrieb: >>> >>> Now we need a nice example of the use of Octave to illustrate the >>> yet-to-be-written entry for the User's Guide. Any suggestions? >> Well as Jack mentioned, maybe (cross-) spectral stuff would be a good >> area because of complex numbers. For example the "cohesion" measure by >> Croux/Forni/Reichlin? > > Thanks for the suggestion. I was kinda hoping that somebody who > knows Octave better than I might actually contribute an example > for the manual. The only things that I know how to do in Octave > are boring stuff that you could do just as well in gretl. The Matlab code is on www.economia.unimore.it/forni_mario/matlab.htm -- I could take a look next week whether that also works on octave and make a "foreign" example. Actually to have the standard coherence available in gretl via this wrapper would be nice. (cohesion would be nice, too, of course, but coherence is standard spectral stuff.)

pkg install specfun-1.0.8.tar.gz pkg install signal-1.0.10.tar.gz

Allin Cottrell schrieb: > 1) In Octave, ensure that the "specfun" and "signal" packages are > installed [...] > > 2) Get Mario Forni's code [...] > > 3) Sample gretl script: > > <script> > open data9-7 > matrix xy = { PRIME, UNEMP } > mwrite(xy, &quot;(a)dotdir/xy.mat&quot;) > > foreign language=octave > xy = gretl_loadmat("xy.mat"); > x = xy(:,1); > y = xy(:,2); > # the last param is the Bartlett window size > h = coher(x, y, 8); > gretl_export(h, "h.mat"); > end foreign > > h = mread(&quot;(a)dotdir/h.mat&quot;) > colnames(h, "coherence") > gnuplot 1 --time --with-lines --matrix=h --output=display > </script>

Hey Allin, I just compiled the new cvs, installed the octave packages via synaptic on my ubuntu system and ran the script. I get the following error message: "octave exited with status 256error: `coher' undefined near line 6 column 5 error: evaluating assignment expression near line 6, column 3 error: near line 6 of file `/home/artur/.gretl/gretltmp.m'" Any ideas what the problem is??

Allin Cottrell schrieb: > On Thu, 25 Feb 2010, Sven Schreiber wrote: > > >> Allin Cottrell schrieb: >> >>> On Wed, 24 Feb 2010, Sven Schreiber wrote: >>> >>> >>>> Allin Cottrell schrieb: >>>> >>>>> Now we need a nice example of the use of Octave to illustrate the >>>>> yet-to-be-written entry for the User's Guide. Any suggestions? >>>>> >>>> Well as Jack mentioned, maybe (cross-) spectral stuff would be a good >>>> area because of complex numbers. For example the "cohesion" measure by >>>> Croux/Forni/Reichlin? >>>> >>> Thanks for the suggestion. I was kinda hoping that somebody who >>> knows Octave better than I might actually contribute an example >>> for the manual. The only things that I know how to do in Octave >>> are boring stuff that you could do just as well in gretl. >>> >> The Matlab code is on www.economia.unimore.it/forni_mario/matlab.htm -- >> I could take a look next week whether that also works on octave and make >> a "foreign" example. Actually to have the standard coherence available >> in gretl via this wrapper would be nice. (cohesion would be nice, too, >> of course, but coherence is standard spectral stuff.) >> > Thanks, that was enought of a hint to get me started. Could anyone > check that the following is correct? > > 1) In Octave, ensure that the "specfun" and "signal" packages are > installed. I did: > > >> pkg install specfun-1.0.8.tar.gz >> pkg install signal-1.0.10.tar.gz >> > (after downloading these packages from octave-forge). > > 2) Get Mario Forni's code. Here's a shell script that downloads > the required files and strips the trailing Ctrl-Z's that prevent > them from working on Linux: > > <bash> > SITE=http://www.economia.unimore.it/forni_mario/MYPROG > > for f in \ > coheren2.m \ > coheren.m \ > coher.m \ > cospec.m \ > crosscov.m \ > crosspec.m \ > crosspe.m \ > spec.m ; do > wget $SITE/$f && \ > cat $f | tr -d \\032 > tmp.m && mv tmp.m $f > done > </bash> > > 3) Sample gretl script: > > <script> > open data9-7 > matrix xy = { PRIME, UNEMP } > mwrite(xy, &quot;(a)dotdir/xy.mat&quot;) > > foreign language=octave > xy = gretl_loadmat("xy.mat"); > x = xy(:,1); > y = xy(:,2); > # the last param is the Bartlett window size > h = coher(x, y, 8); > gretl_export(h, "h.mat"); > end foreign > > h = mread(&quot;(a)dotdir/h.mat&quot;) > colnames(h, "coherence") > gnuplot 1 --time --with-lines --matrix=h --output=display > </script> > > Allin > _______________________________________________ > Gretl-devel mailing list > Gretl-devel(a)lists.wfu.edu > http://lists.wfu.edu/mailman/listinfo/gretl-devel > Hey Allin, I just compiled the new cvs, installed the octave packages via synaptic on my ubuntu system and ran the script. I get the following error message: "octave exited with status 256error: `coher' undefined near line 6 column 5 error: evaluating assignment expression near line 6, column 3 error: near line 6 of file `/home/artur/.gretl/gretltmp.m'" Any ideas what the problem is??

On Thu, 25 Feb 2010, Sven Schreiber wrote: > Allin Cottrell schrieb: >> On Wed, 24 Feb 2010, Sven Schreiber wrote: >> >>> Allin Cottrell schrieb: >>>> Now we need a nice example of the use of Octave to illustrate the >>>> yet-to-be-written entry for the User's Guide. Any suggestions? >>> Well as Jack mentioned, maybe (cross-) spectral stuff would be a good >>> area because of complex numbers. For example the "cohesion" measure by >>> Croux/Forni/Reichlin? >> Thanks for the suggestion. I was kinda hoping that somebody who >> knows Octave better than I might actually contribute an example >> for the manual. The only things that I know how to do in Octave >> are boring stuff that you could do just as well in gretl. > The Matlab code is on www.economia.unimore.it/forni_mario/matlab.htm -- > I could take a look next week whether that also works on octave and make > a "foreign" example. Actually to have the standard coherence available > in gretl via this wrapper would be nice. (cohesion would be nice, too, > of course, but coherence is standard spectral stuff.) Thanks, that was enought of a hint to get me started. Could anyone check that the following is correct?

mwrite(xy, &quot;(a)dotdir/xy.mat&quot;)

colnames(h, "coherence")

gnuplot 1 --time --with-lines --matrix=h --output=display

On Thu, 25 Feb 2010, Sven Schreiber wrote: > Allin Cottrell schrieb: > >> mwrite(xy, &quot;(a)dotdir/xy.mat&quot;) > > OT: I know about the return value of mwrite(), but this "feels" more > like a command than a function. Or to put it differently: if I > understand correctly you recently introduced more sophisticated error > handling ($error and stuff). I wonder whether a mish-mash is being born > right now, sometimes getting error codes from function return values, > and sometimes from accessor variables? I take your point that mwrite feels more like a command than a function. One reason it was done as a function is so that it could return an error code (rather than just bombing out on failure). Now that we have "catch" (see the manual) that could be reconsidered; but symmetry with mread (which is "naturally" a function) would argue for function status for mwrite.

>> colnames(h, "coherence") > > another function used like a command -- how many are there?

In this case, admittedly, there's not much justification for function status -- other than a feeling of mine (anyone else?) that it's better to keep the command namespace of manageable size. I'm always somewhat reluctant to add a new command, but don't feel quite the same constraint with functions.

On Thu, 25 Feb 2010, Riccardo (Jack) Lucchetti wrote: > The gretl scripting language has grown through time > considerably: the lack of coherence, if you will, is a > consequence of the fact that it had started more as a > convenience "batch" language to automate certain jobs, and > gradually has become more and more similar to a programming > language. Of course, since Allin and (to a much lesser extent) > I were the people who put the features in, IMO it's only natural > that the closer you come to a programming language proper, the > more it looks like the programming language that we like the > most (ie C), where _literally everything_ is a function. Yes, exactly. But as we make gretl-script more C-like at the margin, I'm very much aware that while this is to my liking it is not very attractive to people with no programming background.

We could achieve consistency by scrapping "commands" and making everything a function, but I wouldn't favor that (and I don't think Jack is suggesting it).

The "command" mode is more relaxed and less persnickety (less need for commas and parentheses and such syntactical apparatus; more flexibility in the order of arguments -- e.g. option flags can go anywhere).

Yes I think there are two different issues here: one is commands vs. functions, and the other is how to do error handling. With respect to the error handling one could argue to have only one way of doing things. That would raise the question of what functions like mwrite() should return instead.

But mwrite() isn't a command and Allin and Jack gave convincing (IMO) reasons for that...

I'm not sure exactly what should be done here, but I tend to agree that it would be messy to have "catch" apply to function calls, and I'll work on fixing that before the next release. We have a natural substitute in various matrix functions that return a null matrix on failure rather than flagging an error.

>>> In this case, admittedly, there's not much justification for >>> function status -- other than a feeling of mine (anyone else?) >>> that it's better to keep the command namespace of manageable size. >>> I'm always somewhat reluctant to add a new command, but don't feel >>> quite the same constraint with functions. >> 100% agree. > Yes I think that's wise. However, I'm not sure I understand exactly in > some cases why something is a command rather than a function then: > > * criteria It's gone in CVS, replaced by a packaged function.

> * pvalue (although I see the analogy to the menu entry) > * (some of the transformation commands, although I see the case for > user-friendliness there) > * modprint (requires matrix stuff, so arguably is advanced anyway) > * sprintf (recent and advanced, and has natural return value) Some of these, I think, are simply "legacy". There's no real need to have sprintf, for example, in both command and function form: it's just that the command came first. Depending on how concerned we are with backward compatibility we could get rid of some of these.

Where possible, yes. However, some specialized things are implemented as functions, for example the (new) command log entries corresponding to the GUI filter menu items. And that raises another wiki-related point: do we want to scrap the "set" variables for the HP and BK filters now that we can do the equivalent via arguments for hpfilt() and bkfilt()?

On Mon, 1 Mar 2010, Sven Schreiber wrote: > Allin Cottrell schrieb: > > >> Where possible, yes. However, some specialized things are >> implemented as functions, for example the (new) command log >> entries corresponding to the GUI filter menu items. And that >> raises another wiki-related point: do we want to scrap the "set" >> variables for the HP and BK filters now that we can do the >> equivalent via arguments for hpfilt() and bkfilt()? >> > > IMO if the commands are kept then the 'set' variables should also be > kept. Or scrap the commands as well. The HP/BK-related set variables have no effect on _any_ command. Their presence is a relic from a time when multi-argument functions were a coding challenge (we're talking before 2007). If we axe them, the mourning on my side will be very limited.

On Mon, 1 Mar 2010, Riccardo (Jack) Lucchetti wrote: > The HP/BK-related set variables have no effect on _any_ command. Their > presence is a relic from a time when multi-argument functions were a > coding challenge (we're talking before 2007). If we axe them, the > mourning on my side will be very limited. OK, we seem to have a consensus on that. In the next release, or with some "time to digest the deprecation"?

Riccardo (Jack) Lucchetti schrieb: > I'd be more ambitious. > > We now have a number of ideas for backward-incompatible changes, the > most far-reaching being the abolition of "end if" and "end loop". If we > have to break people's scripts, let's do it once and for all. My > proposal: release 1.9.0 advertising it as a preparation release for > 2.0.0. The visible changes would be, apart from what is now in CVS, the > fact that we advertise in the loudest and most annoying way that the > syntax is about to change, via warnings, error messages, pop-up windows, > you name it. I would also suggest to help users in the transition as much as possible. This includes of course documenting the changes (but that is being done nicely anyway), and a tool like Python's '2to3' which tries to automate the transition from python 2.x to python 3.x as far as can be done. I hope I'm not promising too much but I could volunteer to write a Python script that would try to do that with the transition gretl 1.x -> gretl 2.x. (Some parts would be easy, like replacing 'end if' with 'endif' -- that's what you mean by "abolition", right?-- but other parts of course would be more difficult or even impossible.)

> > In the meantime, development continues as normal. When we reach the > following milestones: > > - System GMM estimator for dynamic panel data (Blundell-Bond) > - At least EGARCH/GJR > - Bivariate probit / IV probit > > which are IMO the main holes in our coverage, we release 2.0.0 with the > new syntax as well. I think there is no reason to combine "feature milestones" with the release of 2.0.0. IMHO this just makes it more difficult to communicate to the users the backwards-incompatible changes. New features are being added to gretl all the time (thanks to you!) with just regular minor releases. Why change that now? So maybe those estimators will be ready to be included in the 1.9.x series, or in the 2.0.x series, I think it doesn't matter. (Since they are not coded in gretl script, the syntax changes shouldn't affect them, right?)

(I still don't buy the underlying argument, though...)

Allin Cottrell schrieb: > On Wed, 3 Mar 2010, Sven Schreiber wrote: > > [Sorry, but I'll skip the > 1 quotation levels, since they get a > bit confusing here.] > >> (I still don't buy the underlying argument, though...) > > I think you may have mistaken Jack's point. I took him to be > saying that a "milestone release" that is mostly just backward > incompatible changes is likely to please few people and annoy > quite a number. Sure; that's why I think help may be needed to ease the transition by automating it as far as possible. BTW, I don't understand yet why it's necessary to replace "end if", but I guess that's for another thread.

Personally I have not been able to remember which of the "end" tags need a blank and which do not. To my brain "end if" is like "end function" and I guess I would produce a lot of coding errors in the future if "end if" were not allowed anymore. I suspect that this problem would bite many other users as well. (But of course I may be wrong.) Note that my argument is purely "psychic", but may be important nevertheless.

I should say that I'm not really making this argument on my own account -- I'm not involved in the teaching of econometrics at the postgraduate level -- but I'm willing to assume that Jack's impressions are valid unless proved otherwise.

[A]t some point the two menu items under the /Variable/Spectrum menu seem to have become confused (presumably as a side effect of some other change). The "Sample periodogram" item should not have a window-size selector. That should be fixed soon.

That seems to leave two possibilities: 1) make 1.9 the syntax-"breaking" release, followed by a super-duper 2.0 feature release 2) roll everything into 2.0 Option (1) would have the advantage to get a chance to iron out the bugs coming from the massive underlying changes in 1.9 to have a killer 2.0 version. It would also buy time to actually get the features ready for 2.0. Option (2) would be better in the sense that it's easier to communicate that syntax was changed in version 2.0. Jack: Why do you think gretl needs a feature-laden 2.0 this year? Of course it would be cool, but you sound as if it's a matter of life and death. I'm not convinced, but as you saw before, that might change.

Allin Cottrell schrieb: > On Wed, 24 Feb 2010, Sven Schreiber wrote: > >> And slightly OT: As a pythonista I would of course be interested in >> adding numerical Python (NumPy) to the list of supported foreign >> languages. I'm ready to help with the import/export functions. What >> exactly would be needed? > Basically, two functions in the target language, one to import > gretl matrices as written by mwrite(), and one to export matrices > in the format readable by mread(). Plus one auxiliary function to > return the gretl "dotdir". These get written out by gretl and > somehow (depending on the language) get "sourced" at startup > of "foreign" execution. Ok that should be fairly easy. I may well have done something like that already back when I cooked up "py4gretl". I'll also check that next week.