My comments were made in the context of the inclusion of some thing like STEPLS in gretl. As far as I know STEPLS is an eviews instruction which implements stepwise regression with various options (t-stats, p-values, forward selection, backward selection, combinatorial selections etc.). It does not cover the more recent methods mentioned by Sven. My advice to any economist who wishes to use a procedure such as the eviews STEPLS is simply don't as it never produces valid inference. OLS and many other econometric techniques can be abused but when used properly do produce valid inference. My own opinion is that the inclusion of these stepwise procedures in various econometric packages has, over the years, arisen to some extent from the demands of those who do not understand the difficulties. Perhaps I feel strongly about this because I have been outvoted on several occasions by economists after invalid inferences had been made using stepwise procedures. While I mentioned LSE (Hendry's) general to specific (GETS) I am not advocating a return to strict LSE econometric methodology. At least the initial steps (as I understand it) are based on economic theory and it is a methodology of model selection and validation rather than one of model search, Validation of models uncovered by search techniques must be validated by a new and independent data sample. Generally such a new sample is not available in economics as therefore validation of models uncovered by search techniques is not possible

I would add that the eviews manual section for STEPLS ends with the warning(http://www.eviews.com/help/helpintro.html#page/content%2FRegress2-Stepwise_Least_Squares_Regression.html%23wwconnect_header)

"
Invalid inference is but one of the reasons that stepwise regression and other variable selection methods have a large number of critics amongst statisticians. Other problems include an upwardly biased final R-squared, possibly upwardly biased coefficient estimates, and narrow confidence intervals. It is also often pointed out that the selection methods themselves use statistics that do not account for the selection process."

John C Frain

3 Aranleigh ParkRathfarnham

Dublin 14

Ireland

www.tcd.ie/Economics/staff/frainj/home.html

mailto:frainj@tcd.ie

mailto:frainj@gmail.com

Dublin 14

Ireland

www.tcd.ie/Economics/staff/frainj/home.html

mailto:frainj@tcd.ie

mailto:frainj@gmail.com

On Sun, 17 Jun 2018 at 16:11, Sven Schreiber <svetosch@gmx.net> wrote:

Am 17.06.2018 um 09:03 schrieb Riccardo (Jack) Lucchetti:

> On Sat, 16 Jun 2018, John C Frain wrote:

>

>> Can you give me one good example in which a stepwise regression program

>> produces results that are not subject to the objections mentioned by

>> Clive. OLS, VARS etc may be abused by some but such routines are the

>> basis

>> of much good work. Stepwise regression is not.

>

> On a personal level, I agree 100%. As Clive remarked, the statistical

> issues implied by automatic model selection are indeed thorny. The

> only convincing framework in which you can (to some extent) substitute

> human judgement with CPU cycles is BMA, but apart from that I myself

> would never base any critical statistical procedure on tools like

> stepwise regression.

Let me just remember everybody that this discussion is not only about

the stepwise method. For example, the latest code examples here on the

list were about what you might call best subset, by trying out every

combination (brute force, not elegant).

There are a lot of more recent estimation methods out there that tackle

shrinkage / penalized estimation / model reduction. There are

well-established reasons and a large literature out there justifying

shrinkage methods - where justifying does _not_ mean in all situations,

nor that they cannot be abused of course, and yes proper inference is at

least tricky and quite often unknown.

>

> Having said that, however, I believe that the purpose of the gretl

> project is not to tell the world how econometrics should be done.

> research context, you may want to replicate other people results, and

> do so by using free tools. Gretl must be up to the job, and therefore

> I'm all in favour of implementing statistical techniques I would never

> trust myself.

>

Very well put I think. Because if you think it through, otherwise we

couldn't offer OLS because it is biased in many situations.

cheers,

sven

_______________________________________________

Gretl-users mailing list

Gretl-users@lists.wfu.edu

http://lists.wfu.edu/mailman/listinfo/gretl-users