3:37 p.m.
Dear all,
I just trying to run some forecasts based on an ordered probit
regression model and detected some (maybe?) slight inconsistency -- if any.
Using the fcast command returns for the BINARY PROBIT the estimated
probabilties, but for the ORDERED PROBIT a series of the predicted
response at each observation (the response for which the estimated
probability is highest).
However, I would like to estimate using the ORDER PROBIT the forecasted
probabilties using Jack's "oprobit_predict" function for which I would
need forecast of the z-hat values.
Here is an example:
<hansl>
clear
set messages off
set echo off
open ukppp.gdt -q
series B = (diff(e12)>0) # binary
series C = 0 # ordered (3 classes)
C = (diff(e12)>=-0.02 && diff(e12)<0.07) + 2*(diff(e12)>=0.07)
# Set tranining sample
smpl ; 1986:1
# Binary Probit
probit B 0 doilp0 --quiet
fcast --out-of-sample
# Ordered Probit
probit C 0 doilp0 --quiet
fcast --out-of-sample
</hansl>
Best,
Artur
4:09 p.m.
Am 16.12.2015 um 16:37 schrieb Artur Tarassow:
Dear all,
I just trying to run some forecasts based on an ordered probit
regression model and detected some (maybe?) slight inconsistency -- if any.
Using the fcast command returns for the BINARY PROBIT the estimated
probabilties, but for the ORDERED PROBIT a series of the predicted
response at each observation (the response for which the estimated
probability is highest).
I had nothing to do with the design, but it seems to make sense: For a
binary probit, giving the prob p_1 of the case 1 tells you everything,
because logically the other case must have 1-p_1. But with an ordered
probit you need at least two probs, and you cannot fit this information
into a single series.
Ok you could say for consistency the binary probit then also should give
just the predicted 0 and 1 after fcast.
Have you tried looking what's in the $fcast accessor? Maybe somebody
buried an easter egg in there for ordered probits.
good luck,
sven
4:22 p.m.
Thanks for the response, Sven. $fcast provides (as expected) the same
information.
The point is that I need for the computation of the out-of-sample
probabilities A) the cut-values (provided via $coeff) as well as the
forecasted "z-hat values" which I am not aware of on how to access these.
Artur
Am 16.12.2015 um 17:09 schrieb Sven Schreiber:
Am 16.12.2015 um 16:37 schrieb Artur Tarassow:
> Dear all,
>
> I just trying to run some forecasts based on an ordered probit
> regression model and detected some (maybe?) slight inconsistency -- if
> any.
>
> Using the fcast command returns for the BINARY PROBIT the estimated
> probabilties, but for the ORDERED PROBIT a series of the predicted
> response at each observation (the response for which the estimated
> probability is highest).
I had nothing to do with the design, but it seems to make sense: For a
binary probit, giving the prob p_1 of the case 1 tells you everything,
because logically the other case must have 1-p_1. But with an ordered
probit you need at least two probs, and you cannot fit this information
into a single series.
Ok you could say for consistency the binary probit then also should give
just the predicted 0 and 1 after fcast.
Have you tried looking what's in the $fcast accessor? Maybe somebody
buried an easter egg in there for ordered probits.
good luck,
sven
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
4:44 p.m.
Am 16.12.2015 um 17:22 schrieb Artur Tarassow:
Thanks for the response, Sven. $fcast provides (as expected) the
same
information.
The point is that I need for the computation of the out-of-sample
probabilities A) the cut-values (provided via $coeff) as well as the
forecasted "z-hat values" which I am not aware of on how to access these.
I don't have much experience (=none?) with these type of models, but
from looking at section 32.2 of the user guide it would seem to me:
a) \hat{\beta} is available via $coeff
b) The X-es are available as observables for forecasting.
c) So you can get \hat{z} = X \hat{\beta}
d) The alphas (cuts) are also available in $coeff
And that's all you need, no? But maybe I'm missing something.
cheers,
sven
4:44 p.m.
On Wed, 16 Dec 2015, Artur Tarassow wrote:
Thanks for the response, Sven. $fcast provides (as expected) the same
information.
The point is that I need for the computation of the out-of-sample
probabilities A) the cut-values (provided via $coeff) as well as the
forecasted "z-hat values" which I am not aware of on how to access these.
The z-hat values are just X\hat{beta}.
probit y X
X -= const
series zhat = {X}*$coeff[1:nelem(X)]
Allin
4:57 p.m.
On Wed, 16 Dec 2015, Artur Tarassow wrote:
Dear all,
I just trying to run some forecasts based on an ordered probit regression
model and detected some (maybe?) slight inconsistency -- if any.
Using the fcast command returns for the BINARY PROBIT the estimated
probabilties, but for the ORDERED PROBIT a series of the predicted response
at each observation (the response for which the estimated probability is
highest).
However, I would like to estimate using the ORDER PROBIT the forecasted
probabilties using Jack's "oprobit_predict" function for which I would need
forecast of the z-hat values.
oprobit_predict is by Allin (credit where it's due ;) )
That said: here's a slight modification of your script:
<hansl>
set messages off
set echo off
include oprobit_predict.gfn
open ukppp.gdt -q
series C = 0 # ordered (3 classes)
C = (diff(e12)>=-0.02 && diff(e12)<0.07) + 2*(diff(e12)>=0.07)
list X = 0 doilp0
# Set tranining sample
smpl ; 1986:1
# Ordered Probit
probit C X #--quiet
smpl full
list Z = X - const
scalar k = nelem(Z)
series zhat = lincomb(Z, $coeff[1:k])
matrix cut = $coeff[k+1:]
# get the predicted responses
series pred = oprobit_pred(cut, zhat)
</hansl>
Now, having sorted the mechanical aspect out, a more interesting problem
remains open: how do you evaluate forecasts in a case like this? For
continuous outcomes it makes sense to rely on some relatively
straightforward loss functions (also, take a look at Giulio Palomba's
package for the Diebold-Mariano test). But how do you even _define_ the
loss function for a discrete outcome? Hmmm... mumble, mumble...
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:27 p.m.
<But how do you even _define_ the
loss function for a discrete outcome? Hmmm... mumble, mumble...
Something on binary outcomes should be here
http://www.albany.edu/economics/research/workingp/2012/lahiriyang.pdf
Oleh
9:43 a.m.
Thank you for all the quick replies.
Am 16.12.2015 um 17:57 schrieb Riccardo (Jack) Lucchetti:
oprobit_predict is by Allin (credit where it's due ;) )
Definitely: Thank you Allin for providing this nice and lean function! ;-)
That said: here's a slight modification of your script:
Thanks for modifying this, Jack. It works nicely.
Now, having sorted the mechanical aspect out, a more interesting
problem
remains open: how do you evaluate forecasts in a case like this? For
continuous outcomes it makes sense to rely on some relatively
straightforward loss functions (also, take a look at Giulio Palomba's
package for the Diebold-Mariano test). But how do you even _define_ the
loss function for a discrete outcome? Hmmm... mumble, mumble...
This is definitely the next start. Actually, I've got Giulio's package
already on the list for this. Also Oleg's reference to the Handbook
article is on the desk for reading between Christmas and New Year :-)
Best,
Artur
6:43 p.m.
Dear Arthur, Predicting probabilties for odered probit are very easy to be user-written,
one can use
oprobit_predict as s helper; the more interesting thing is what to do with confidence
intervals: for ordinary probit the probs are F(x'b), where F() is monotonous, so we
could compute an interval for x'b, say [i_1,i_2], and use F( [i_1,i_2] ); for ordered
probit using delta-method could give interval ends outside of [0;1] Oleh
16 грудня 2015, 17:37:11, від "Artur Tarassow" <
artur.tarassow(a)googlemail.com >:
Dear all,
I just trying to run some forecasts based on an ordered probit
regression model and detected some (maybe?) slight inconsistency -- if any.
Using the fcast command returns for the BINARY PROBIT the estimated
probabilties, but for the ORDERED PROBIT a series of the predicted
response at each observation (the response for which the estimated
probability is highest).
However, I would like to estimate using the ORDER PROBIT the forecasted
probabilties using Jack's "oprobit_predict" function for which I would
need forecast of the z-hat values.
Here is an example:
<hansl>
clear
set messages off
set echo off
open ukppp.gdt -q
series B = (diff(e12)>0) # binary
series C = 0 # ordered (3 classes)
C = (diff(e12)>=-0.02 && diff(e12)<0.07) + 2*(diff(e12)>=0.07)
# Set tranining sample
smpl ; 1986:1
# Binary Probit
probit B 0 doilp0 --quiet
fcast --out-of-sample
# Ordered Probit
probit C 0 doilp0 --quiet
fcast --out-of-sample
</hansl>
Best,
Artur
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
7:04 p.m.
New subject: Forecast based on ordered probit: a promise
In several days my package for out-of sample probabilities with bootstrapping CI for
ordered logit and probit and multinomial logit will be upload by me to the staging area.
Oleh
16 грудня 2015, 17:37:11, від "Artur Tarassow" <
artur.tarassow(a)googlemail.com >:
Dear all,
I just trying to run some forecasts based on an ordered probit
regression model and detected some (maybe?) slight inconsistency -- if any.
Using the fcast command returns for the BINARY PROBIT the estimated
probabilties, but for the ORDERED PROBIT a series of the predicted
response at each observation (the response for which the estimated
probability is highest).
However, I would like to estimate using the ORDER PROBIT the forecasted
probabilties using Jack's "oprobit_predict" function for which I would
need forecast of the z-hat values.
Here is an example:
<hansl>
clear
set messages off
set echo off
open ukppp.gdt -q
series B = (diff(e12)>0) # binary
series C = 0 # ordered (3 classes)
C = (diff(e12)>=-0.02 && diff(e12)<0.07) + 2*(diff(e12)>=0.07)
# Set tranining sample
smpl ; 1986:1
# Binary Probit
probit B 0 doilp0 --quiet
fcast --out-of-sample
# Ordered Probit
probit C 0 doilp0 --quiet
fcast --out-of-sample
</hansl>
Best,
Artur
_______________________________________________
Gretl-devel mailing list
Gretl-devel(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-devel
3051
days inactive
3052
days old
9 comments
5 participants
participants (5)
-
Allin Cottrell -
Artur Tarassow -
oleg_komashko＠ukr.net -
Riccardo (Jack) Lucchetti -
Sven Schreiber