Am 15.06.2018 um 14:55 schrieb Riccardo (Jack) Lucchetti: Thanks for pointing it out, it wasn't obvious to me before that it can be viewed as a variable selection tool. There is also the 'addlist' function package by Allin which does the specific-to-general direction. (omit --auto being general-to-specific) With "combinatorial" do you mean trying out all possible specifications by brute force? cheers, sven

​​ ​​ ​​ ​​ ​It would be nice if both these approaches and possibly the combinatorial one where included ​ in a menu item "model". It will be easy for the average use and students to use. PG *Periklis Gogas <http://www.econ.duth.gr/personel/dep/gkogkas/index.en.shtml>* Associate Professor of Economic Analysis and International Economics Department of Economics, Democritus University of Thrace Associate Editor - Journal of Economic Asymmetries <https://www.journals.elsevier.com/the-journal-of-economic-asymmetries/> Euro Area Business Cycle Network - Fellow <http://www.eabcn.org/person/periklis-gogas> The Rimini Centre for Economic Analysis - Fellow <http://www.rcfea.org/component/option,com_frontpage/Itemid,1/> The Society for Economic Measurement - Member <http://sem.society.cmu.edu/home.html> Institute for Nonlinear Dynamical Inference (INDI) - Charter Fellow <http://icemr.ru/institute-for-nonlinear-dynamical-inference/> Σύγχρονοι Ελληνικοί Μύθοι <http://www.public.gr/product/syghronoi-ellinikoi-mythoi/prod9040056pp/?so... - το νέο μου βιβλίο On Fri, Jun 15, 2018 at 5:02 PM Periklis Gogas <perrygogas(a)gmail.com&gt; wrote:

Em 15 de junho de 2018, Periklis Gogas escreveu: Yes I mean all possible combinations of regressors. Dear Periklis, I have an almost finishe function called Automod that tests all the possible combinations. If you want to take a look I can share with you. Best, Henrique Andrade

Some econometric software packages have model search tools such as those proposed here. During my working days I have seen tools such as these being misused. I would be sure that no one on this list would use such a tool to search for a model that "fitted" the data well and then present the final t-stats etc, as if they were the results of the estimation of some model derived from theory. This is either dishonest or incompetent. On several occasions when presented with such work I tried to explain the statistical problems to be told that if there were such problems the routines would not have been included in the software. My recommendation would be not to include such a routine in gretl. Gretl is an excellent econometric package and we should avoid introducing routines that bring econometrics into disrepute. John C Frain 3 Aranleigh Park Rathfarnham Dublin 14 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj@tcd.ie mailto:frainj@gmail.com On Fri, 15 Jun 2018 at 18:58, Periklis Gogas <perrygogas(a)gmail.com&gt; wrote:

Am 15.06.2018 um 21:57 schrieb Clive Nicholas: I agree maybe about 95% :-) The statistical points you remind us of are of course valid. Still that doesn't mean that a formalized model specification search is never to be used. The question is indeed how to communicate that problem in a software package (including the existing omit --auto option). Perhaps the result of such a procedure should be presented wihtout standard errors or something like that. An easier solution would probably be to include a warning message. BTW, the Lasso and other modern tools haven't been mentioned yet. Indeed I wouldn't currently recommend to try to reinvent the wheel in hansl. It is relatively easy to wrap the R packages glmnet and sparsenet in a gretl function. cheers, sven

Perhaps there is some justification in model reduction as implemented in David Hendry's Pc-gets or in the Grocer package in Scilab. This does use a form of model reduction but it is very restricted. Otherwise what are the benefits of systems of stepwise regression. If you do arrive a a somewhat sensible looking answer you can make no structural econometric inference. How do you overcome the problems mentioned by Clive Nicolas. You then need a completely new and independent data set to make statistical inferences on the new theory. Perhaps you are simply interested in data reduction. Can you give me one good example in which a stepwise regression program produces results that are not subject to the objections mentioned by Clive. OLS, VARS etc may be abused by some but such routines are the basis of much good work. Stepwise regression is not. John C Frain 3 Aranleigh Park Rathfarnham Dublin 14 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj@tcd.ie mailto:frainj@gmail.com On Fri, 15 Jun 2018 at 22:36, Periklis Gogas <perrygogas(a)gmail.com&gt; wrote:

On 17 June 2018 at 08:03, Riccardo (Jack) Lucchetti <r.lucchetti(a)univpm.it&gt; wrote: Having said that, however, I believe that the purpose of the gretl project If the book you're referring to is "Mostly Harmless Econometrics" by Josh Angrist and Steve Pischke, I tend to agree. Although there is some good stuff in it, their treatment of IV regression models is uncritical yet complex to grasp and their view of which techniques should go into an econometrican's toolbox is, well, myopic to say the least: they talk throughout parts of the book of modelling data from schools, classrooms and pupils and yet *nowhere* do they mention the value of using multilevel models for such data, not even their merits in general; *ditto* for survival analysis models, even though a sizable part of economics is forever concerned with estimating factors affecting variations in the survival rates of firms in various industries. Moreover, the material they include in their concluding chapter should really have been put near the start. It's well worth a read, but it's a curiously odd book and not one I would recommend to people as a starting point on econometrics. Still, at least it doesn't waste its time discussing stepwise. C

My comments were made in the context of the inclusion of some thing like STEPLS in gretl. As far as I know STEPLS is an eviews instruction which implements stepwise regression with various options (t-stats, p-values, forward selection, backward selection, combinatorial selections etc.). It does not cover the more recent methods mentioned by Sven. My advice to any economist who wishes to use a procedure such as the eviews STEPLS is simply don't as it never produces valid inference. OLS and many other econometric techniques can be abused but when used properly do produce valid inference. My own opinion is that the inclusion of these stepwise procedures in various econometric packages has, over the years, arisen to some extent from the demands of those who do not understand the difficulties. Perhaps I feel strongly about this because I have been outvoted on several occasions by economists after invalid inferences had been made using stepwise procedures. While I mentioned LSE (Hendry's) general to specific (GETS) I am not advocating a return to strict LSE econometric methodology. At least the initial steps (as I understand it) are based on economic theory and it is a methodology of model selection and validation rather than one of model search, Validation of models uncovered by search techniques must be validated by a new and independent data sample. Generally such a new sample is not available in economics as therefore validation of models uncovered by search techniques is not possible I would add that the eviews manual section for STEPLS ends with the warning( http://www.eviews.com/help/helpintro.html#page/content%2FRegress2-Stepwis... ) " Invalid inference is but one of the reasons that stepwise regression and other variable selection methods have a large number of critics amongst statisticians. Other problems include an upwardly biased final R-squared, possibly upwardly biased coefficient estimates, and narrow confidence intervals. It is also often pointed out that the selection methods themselves use statistics that do not account for the selection process." John C Frain 3 Aranleigh Park Rathfarnham Dublin 14 Ireland www.tcd.ie/Economics/staff/frainj/home.html mailto:frainj@tcd.ie mailto:frainj@gmail.com On Sun, 17 Jun 2018 at 16:11, Sven Schreiber <svetosch(a)gmx.net&gt; wrote:

Am 18.06.2018 um 09:12 schrieb Marcin Błażejowski: Great! thanks, sven

Am 15.06.2018 um 22:02 schrieb Henrique Andrade: Hi Henrique, thanks for this code. This could be quite easily turned into a nice function package. What would it take to convince you to do it? cheers, sven

Am 16.06.2018 um 13:04 schrieb Riccardo (Jack) Lucchetti: Thanks Jack. I think one of your two implementations should eventually go into the extra.gfn package. I slightly prefer the second one, but what do you think? I'm attaching some code with a function that uses your AltPSetMat function to do the combinatorial regression. (Henrique, I haven't measured it, but my intuition would be that it is faster to refer to the variables by their ID numbers instead of their names, so that's what is done here.) Something like this could become a function package. And slightly OT about gretl internals: Along the way I've noticed two things. The first is that the deflist() function doesn't accept a vector of series IDs as input, in the way that genr-based list construction like "list L = {1,2}" does; maybe deflist should accept that. The second thing is that it would be nice to have a generalization of the varnum() function, to return a vector of series IDs for a list input. See my workaround helper function listIDs in the file. (This would be a bit like the generalization of varname to varnames, for example.) cheers, sven

On Sat, 16 Jun 2018, Henrique Andrade wrote: I guess that the computational cost of the combinatorics is probably comparable to the actual CPU usage for estimation. For example: <hansl> set verbose off function matrices AltPSetMat(matrix X) if rows(X) == 0 matrices PowerSet = defarray({}) else matrix Set = msortby(uniq(vec(X)), 1) n = rows(Set) N = 2^n matrices PowerSet = array(N) if n == 1 PowerSet[1] = {} PowerSet[2] = Set[1] else tmp = AltPSetMat(Set[1:n-1]) PowerSet = tmp loop i = 1 .. nelem(tmp) --quiet PowerSet += tmp[i] ~ Set[n] endloop endif endif return PowerSet end function open mroz87.gdt --quiet list X = 3 4 5 6 10 11 15 16 18 19 matrix mx = X set stopwatch matrices C = AltPSetMat(mx) t0 = $stopwatch n = nelem(C) bestbic = $huge best = 0 loop i = 1 .. n --quiet xi = C[i] list Z = xi ols WW const Z --quiet if $bic < bestbic best = i bestbic = $aic list bestZ = $xlist endif endloop t1 = $stopwatch print t0 t1 print best n ols WW bestZ </hansl> ------------------------------------------------------- Riccardo (Jack) Lucchetti Dipartimento di Scienze Economiche e Sociali (DiSES) Università Politecnica delle Marche (formerly known as Università di Ancona) r.lucchetti(a)univpm.it http://www2.econ.univpm.it/servizi/hpp/lucchetti -------------------------------------------------------

Am 18.06.2018 um 10:45 schrieb Sven Schreiber: BTW, it seems I was able to crash gretl by cranking up the input length to these power set functions. First the (Ubuntu) system became very unresponsive in the GUI, and after a while all gretl windows disappeared. I guess gretl was requesting too much memory or something like that; maybe some internal checks are needed to catch such a case when a hansl user function is too crazy. thanks, sven