My comments were made in the context of the inclusion of some thing like
STEPLS in gretl. As far as I know STEPLS is an eviews instruction which
implements stepwise regression with various options (t-stats, p-values,
forward selection, backward selection, combinatorial selections etc.). It
does not cover the more recent methods mentioned by Sven. My advice to any
economist who wishes to use a procedure such as the eviews STEPLS is simply
don't as it never produces valid inference. OLS and many other
econometric techniques can be abused but when used properly do produce
valid inference. My own opinion is that the inclusion of these stepwise
procedures in various econometric packages has, over the years, arisen to
some extent from the demands of those who do not understand the
difficulties. Perhaps I feel strongly about this because I have been
outvoted on several occasions by economists after invalid inferences had
been made using stepwise procedures. While I mentioned LSE (Hendry's)
general to specific (GETS) I am not advocating a return to strict LSE
econometric methodology. At least the initial steps (as I understand it)
are based on economic theory and it is a methodology of model selection and
validation rather than one of model search, Validation of models uncovered
by search techniques must be validated by a new and independent data
sample. Generally such a new sample is not available in economics as
therefore validation of models uncovered by search techniques is not
possible
I would add that the eviews manual section for STEPLS ends with the warning(
http://www.eviews.com/help/helpintro.html#page/content%2FRegress2-Stepwis...
)
" Invalid inference is but one of the reasons that stepwise regression and
other variable selection methods have a large number of critics amongst
statisticians. Other problems include an upwardly biased final R-squared,
possibly upwardly biased coefficient estimates, and narrow confidence
intervals. It is also often pointed out that the selection methods
themselves use statistics that do not account for the selection process."
John C Frain
3 Aranleigh Park
Rathfarnham
Dublin 14
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
On Sun, 17 Jun 2018 at 16:11, Sven Schreiber <svetosch(a)gmx.net> wrote:
Am 17.06.2018 um 09:03 schrieb Riccardo (Jack) Lucchetti:
> On Sat, 16 Jun 2018, John C Frain wrote:
>
>> Can you give me one good example in which a stepwise regression program
>> produces results that are not subject to the objections mentioned by
>> Clive. OLS, VARS etc may be abused by some but such routines are the
>> basis
>> of much good work. Stepwise regression is not.
>
> On a personal level, I agree 100%. As Clive remarked, the statistical
> issues implied by automatic model selection are indeed thorny. The
> only convincing framework in which you can (to some extent) substitute
> human judgement with CPU cycles is BMA, but apart from that I myself
> would never base any critical statistical procedure on tools like
> stepwise regression.
Let me just remember everybody that this discussion is not only about
the stepwise method. For example, the latest code examples here on the
list were about what you might call best subset, by trying out every
combination (brute force, not elegant).
There are a lot of more recent estimation methods out there that tackle
shrinkage / penalized estimation / model reduction. There are
well-established reasons and a large literature out there justifying
shrinkage methods - where justifying does _not_ mean in all situations,
nor that they cannot be abused of course, and yes proper inference is at
least tricky and quite often unknown.
>
> Having said that, however, I believe that the purpose of the gretl
> project is not to tell the world how econometrics should be done.
> research context, you may want to replicate other people results, and
> do so by using free tools. Gretl must be up to the job, and therefore
> I'm all in favour of implementing statistical techniques I would never
> trust myself.
>
Very well put I think. Because if you think it through, otherwise we
couldn't offer OLS because it is biased in many situations.
cheers,
sven
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users