On Wed, 1 Oct 2014, Artur Bala wrote:
Thank you Jack and Allin,
Jack's right asking "Why should it be anything else?" but I do not have
the
answer actually. I read in the litterature that the 0.5 probability
threshold looks somewhat arbitrary and I was guessing may be gretl is
performing based any other criteria, such as the ROC Curve, in order to
maximize the % of correctly predicted cases.
Of course the "best" criterion depends on what your loss function is. Of
course, the 1/2 threshold can be intuitively motivated by a subjectivist,
de Finetti-like, insurance-oriented reasoning (G. Millo may weigh in at
any time with his vastly superior knowledge on this) such as "if you had
to place a bet". Of course, if you get 1 euro if you win and you pay 1
euro if you lose, the best thing to do is to go with a 0.5 threshold. If
the terms of the bet are different, well...
Stata has some in-build commands for this purpose and the ROC curves
are
generally used to compare the performance of different binary dependent
variable models. As an illustration I expanded Jack's script and found
that maximizing the % of correctly predicted occurs at a threshold of
0.56.
Very, very nice script. How about writing a "ROC curve" function package?
That'd be a very nice addition.
Also: the two commands
<hansl>
gnuplot 1 3 --matrix=result --with-lines --suppress-fitted --output=display \
{set title 'ROC Curve'; set xrange [-0.01:1.01]; set yrange [-0.01:1.01]; set
grid; show grid}
gnuplot 4 5 --matrix=result --with-lines --suppress-fitted --output=display \
{set title 'Correctly predicted %'; set grid; show grid}
</hansl>
are very nice examples of the point I was trying to make a few days ago in
a post to to developers list: there may be scope for introduciong a
backward-incompatible change and drive gnuplot through a dedicated
"foreign" block instead of the present syntax. I'm a bit surprised at the
relative lack of reaction. Any thoughts, anyone?
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------