On Mon, 6 Jul 2009, Riccardo (Jack) Lucchetti wrote: Yes, agreed on both counts. Yes, indeed. I've been thinking about that too. The first point to make is that trying to interpret a "word" as an R function name is the "last resort" in genlex.c. So any built-in, or user-defined, gretl function will mask an R function of the same name. My first thought on this is to the same effect but a little simpler: we could mark off R functions (either built-in, or defined via "foreign" blocks within gretl) by using the prefix "R_". So if you wanted to invoke the R "choose" function you'd have to invoke it as "R_choose(args)". (Internally, we'd see the "R_" and pass the remainder of the word to the new function get_R_function_by_name().) This would reliably avert clashes with gretl built-in functions since gretl uses lower-case identifiers exclusively -- and I don't envisage changing that since I hate mixed-case function names! It does mean that if someone defines a user function "R_foo" it will mask any "foo" function that may exist in R, but that doesn't seems like a big problem to me -- particularly if the manual mentions this point. Yes, it does raise a portability issue. We do have some of those already: for example, the gretl command "mpols" will work only if gretl is linked against (the optional) libgmp. There's also the business of the optional third-party programs TRAMO/SEATS and X-12-ARIMA. At present the functionality of these programs is available only via the GUI, but it probably should be available via script. There are some other issues too. libR is not an "ordinary" library: by plugging into its API you basically "buy into" an R session. And this can be quite lumpy. R seems to make liberal use of the evil "longjmp" function (C's non-local goto -- and you thought that "goto" was bad!). So, for example, the innocuous-looking R function "findFun" crashes the caller on failure. Similarly, a call to "Rf_initEmbeddedR" will abort the caller if the environment is not set up correctly for R. I've tried to guard against both of these issues in recent changes in gretl_foreign.c, but I'm sure there are more. In addition, if you run a gretl/Rlib session under valgrind valgrind --tool=memcheck --leak-check=full --show-reachable=yes you'll see that libR wants to allocate at least 10 MB before it does _anything_. Christoph gave an example function, defined in R, that took a scalar argument and returned its argument plus 1. Fine, R will do that for you if you give it 10 MB of RAM and a several dozen function calls. (OK, any library uses resources; I'm just saying that the libR approach is not a magic bullet for saving on initialization costs.) Allin.

On Mon, 6 Jul 2009, Allin Cottrell wrote: I like this. The only possible problem I can see is parsing the string correctly when a dot is there, as in R_as.matrix() True. And you can add LaTeX on top of that if you want. But you'll agree the ones you mention are rather special cases: mpols is definitely not something you'd normally use for regressions, and the TRAMO/SEATS-X12 stuff is hard to use inadvertently, so to speak. My concern was coming from the idea that if you start mixing native and R functions in your scripts, to the point when it's hard to tell them apart, as in your choose() example, then one day you take your script to another machine where R isn't installed and, bang!, your script is useless. [...] Chris' work and yours have been splendid. Integration with R is now much, much better than it was. Thanks! Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 7 Jul 2009, Talha Yalta wrote: No-one is forcing anyone to use R from within gretl. But you'll agree with me that having the possibility of interacting in the smoothest possible way with R is 1) nice to have: hey, this is free software; if R is better than gretl at something, why not use it? 2) cool, as gretl does R, but R doesn't do gretl :-) 3) attractive for people who like R but would like a friendlier environment 4) totally orthogonal to gretl's future development plans. So I can't see what the downside is. Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 7 Jul 2009, Talha Yalta wrote: I don't really think we need a policy here. Even from a political standpoint (which is not my primary concern anyway), we're not even mixing licences, since both gretl and R are GPL. There may be gretl users who don't even know what R is, people who use it sporadically (like myself), others who may use gretl just as an R frontend. Again, I can't see any problem with any of these attitudes. In my view, gretl's aim should be to make applied econometrics as practical and efficient as possible. If this involves borrowing stuff from other projects, so be it. Let me remind that we've been doing this forever, internally (gretl's source is full of stuff adapted from R, netlib, gnumeric etcetera). It's called "division of labour" :-) The only drawback to this idea is that encapsulating code from other projects may be so impractical to outweigh the benefits. I had a conversation recently with Christian Brownlees, the author of dynamo (http://dynamo.sourceforge.net/), who argued that he'd like to integrate dynamo into gretl, but that implies a dependency on libgsl, which is not on our agenda, at least in the short term (see http://lists.wfu.edu/pipermail/gretl-users/2009-June/003351.html). But in principle cooperation between projects is IMHO the way to go. Well, we've had (c) for a while, as the "foreign" command. For all I know, people are quite happy with it and nothing tragic has happened so far. Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 7 Jul 2009, chris wrote: Yup, I think we're all agreed on that. Yes, it's very nice to be able to send inputs, and grab output, directly. True, but I don't see any harm in exposing things like "choose" and other such relatively simple built-ins. That's certainly worth thinking about. We'll have to come back to this. Allin.

On Tue, 7 Jul 2009, Sven Schreiber wrote: This is an _excellent_ idea. Even better than an "include", we may use our "set" command. Something like set R_support [on|off] with "off" as the default. What do you guys think? Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 7 Jul 2009, Allin Cottrell wrote: Agreed! Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 7 Jul 2009, Allin Cottrell wrote: A few more developments. First, you can pass series arguments to R functions and they'll be converted to matrices. Second, I've tried virtualizing the R functions we use in gretl_foreign.c. I've got this to work on Linux -- not linking against libR.so, but dlopen'ing it instead. There's code in place that should do the same trick on Windows if R.dll is installed, but I haven't managed to test it yet. If anyone would like to have a go with the current snapshot (1.8.2cvs), here's a little test script: <script> open data4-1 set R_functions on foreign language=RLib ols<-function(y,x) { m <- lm(y~x+1) b <- as.matrix(coefficients(m)) invisible(b) } end foreign set messages off matrix rb = R_ols(price, sqft) rb </script> It may be that using dlopen is a good way to go on all platforms, rather than linking to libR at compile time. Allin.