9:13 p.m.
I was discussing with a colleague an empirical application, and he told
me that the only problem he had was that some elements of the Hessian
was "nan" because they came up with the wrong sign. But the R code that
he used did print out the Hessian, with these elements as "nan".
Can we do that in gretl (natively)? I ask because in standard
mle-implementation with the option --hessian we may get "Hessian is not
negative-definite, dropping back to OPG".
What I ask is whether we can instruct gretl to stick with the Hessian,
returning some "nan" values in the diagonal / std errors.
--
Alecos Papadopoulos PhD
Affiliate Researcher
Dpt of Economics, Athens University of Economics and Business
Foundation for Economic and Industrial Research (IOBE)
web: alecospapadopoulos.wordpress.com/
ORCID:0000-0003-2441-4550
1:07 p.m.
Am 23.09.2024 um 23:13 schrieb Alecos Papadopoulos:
I was discussing with a colleague an empirical application, and he
told me that the only problem he had was that some elements of the
Hessian was "nan" because they came up with the wrong sign. But the R
code that he used did print out the Hessian, with these elements as
"nan".
Can we do that in gretl (natively)? I ask because in standard
mle-implementation with the option --hessian we may get "Hessian is
not negative-definite, dropping back to OPG".
If you're in a middle of an
algorithm that uses the Hessian as a
numerical input, I don't see how you can (or should) ignore that this
input is not what you think it is. Just like you wouldn't want to
standardize your data with a standard error estimate that somehow came
out as negative, or would you?
What I ask is whether we can instruct gretl to stick with the Hessian,
returning some "nan" values in the diagonal / std errors.
Not that I'm aware of. Regarding the cited "R code" of your
colleague,
maybe it has a clever solution, maybe it just prints out interim stuff
and stops there, or maybe it does very unwise things; we have no way of
knowing.
cheers
sven
9:52 p.m.
If the hessian is not negative definite or positive definite you have found
a saddle point. You may need to refine your methodology to find the
required minimum or maximum.
When you have found the proper extreme value you can calculate the the
Hessian using numerical derivatives. If calculating the Hessian and its
inverse in R it may be necessary to use high precision arithmetic.
John C Frain
3 Aranleigh Park
Rathfarnham
Dublin 14
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
https://jcfrain.wordpress.com/
https://jcfraincv19.wordpress.com/
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
On Tue, 24 Sept 2024 at 14:08, Sven Schreiber <sven.schreiber(a)fu-berlin.de>
wrote:
Am 23.09.2024 um 23:13 schrieb Alecos Papadopoulos:
> I was discussing with a colleague an empirical application, and he
> told me that the only problem he had was that some elements of the
> Hessian was "nan" because they came up with the wrong sign. But the R
> code that he used did print out the Hessian, with these elements as
> "nan".
>
> Can we do that in gretl (natively)? I ask because in standard
> mle-implementation with the option --hessian we may get "Hessian is
> not negative-definite, dropping back to OPG".
If you're in a middle of an algorithm that uses the Hessian as a
numerical input, I don't see how you can (or should) ignore that this
input is not what you think it is. Just like you wouldn't want to
standardize your data with a standard error estimate that somehow came
out as negative, or would you?
>
> What I ask is whether we can instruct gretl to stick with the Hessian,
> returning some "nan" values in the diagonal / std errors.
>
Not that I'm aware of. Regarding the cited "R code" of your colleague,
maybe it has a clever solution, maybe it just prints out interim stuff
and stops there, or maybe it does very unwise things; we have no way of
knowing.
cheers
sven
_______________________________________________
Gretl-users mailing list -- gretl-users(a)gretlml.univpm.it
To unsubscribe send an email to gretl-users-leave(a)gretlml.univpm.it
Website:
https://gretlml.univpm.it/postorius/lists/gretl-users.gretlml.univpm.it/
10:02 p.m.
Thanks both for the input.
The reason I asked whether we have such functionality in gretl, is not
in order to "go along" with an impaired Hessian, but for diagnostic
purposes: to give a related example, when an mle command fails to
converge, personally it helps me to inspect in detail and trace
iteration-by-iteration how the estimator lost its way. In the same
spirit, obtaining an impaired Hessian, would tell me which coefficients
under estimation are linked to the problem, and this could possibly help
in realizing the source/cause of the failure.
Alecos Papadopoulos PhD
Affiliate Researcher
Dpt of Economics, Athens University of Economics and Business
Foundation for Economic and Industrial Research (IOBE)
web: alecospapadopoulos.wordpress.com/
ORCID:0000-0003-2441-4550
On 26/9/2024 00:52, John C Frain wrote:
If the hessian is not negative definite or positive definite you have
found a saddle point. You may need to refine your methodology to find
the required minimum or maximum.
When you have found the proper extreme value you can calculate the the
Hessian using numerical derivatives. If calculating the Hessian and
its inverse in R it may be necessary to use high precision arithmetic.
John C Frain
3 Aranleigh Park
Rathfarnham
Dublin 14
Ireland
www.tcd.ie/Economics/staff/frainj/home.html
<http://www.tcd.ie/Economics/staff/frainj/home.html>
https://jcfrain.wordpress.com/
https://jcfraincv19.wordpress.com/
mailto:frainj@tcd.ie
mailto:frainj@gmail.com
On Tue, 24 Sept 2024 at 14:08, Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
Am 23.09.2024 um 23:13 schrieb Alecos Papadopoulos:
> I was discussing with a colleague an empirical application, and he
> told me that the only problem he had was that some elements of the
> Hessian was "nan" because they came up with the wrong sign. But
the R
> code that he used did print out the Hessian, with these elements as
> "nan".
>
> Can we do that in gretl (natively)? I ask because in standard
> mle-implementation with the option --hessian we may get "Hessian is
> not negative-definite, dropping back to OPG".
If you're in a middle of an algorithm that uses the Hessian as a
numerical input, I don't see how you can (or should) ignore that this
input is not what you think it is. Just like you wouldn't want to
standardize your data with a standard error estimate that somehow
came
out as negative, or would you?
>
> What I ask is whether we can instruct gretl to stick with the
Hessian,
> returning some "nan" values in the diagonal / std errors.
>
Not that I'm aware of. Regarding the cited "R code" of your
colleague,
maybe it has a clever solution, maybe it just prints out interim
stuff
and stops there, or maybe it does very unwise things; we have no
way of
knowing.
cheers
sven
_______________________________________________
Gretl-users mailing list -- gretl-users(a)gretlml.univpm.it
To unsubscribe send an email to gretl-users-leave(a)gretlml.univpm.it
Website:
https://gretlml.univpm.it/postorius/lists/gretl-users.gretlml.univpm.it/
_______________________________________________
Gretl-users mailing list --gretl-users(a)gretlml.univpm.it
To unsubscribe send an email togretl-users-leave(a)gretlml.univpm.it
Website:https://gretlml.univpm.it/postorius/lists/gretl-users.gretlml.univpm.it/
7:19 a.m.
On 26/09/2024 00:02, Alecos Papadopoulos wrote:
Thanks both for the input.
The reason I asked whether we have such functionality in gretl, is not
in order to "go along" with an impaired Hessian, but for diagnostic
purposes: to give a related example, when an mle command fails to
converge, personally it helps me to inspect in detail and trace
iteration-by-iteration how the estimator lost its way. In the same
spirit, obtaining an impaired Hessian, would tell me which coefficients
under estimation are linked to the problem, and this could possibly help
in realizing the source/cause of the failure.
I understand that what you want is a way to inspect the Hessian after
convergence has taken place. To do this, you may find the "numhess"
function useful. For example, here I use mle to estimate the parameters
of a gamma density and then calculate the Hessian.
<hansl>
function series llik(const series x, scalar theta, scalar p)
series ret = -p * log(theta) - lngamma(p) + (p-1) * log(x) - x/theta
return ret
end function
function scalar total_llik(matrix param, const series x)
series l = llik(x, param[1], param[2])
return sum(l)
end function
nulldata 128
x = -log(uniform())
a = 1
p = 1
mle loglik = llik(x, a, p)
params a p
end mle
theta = {a, p}
H = numhess(theta, "total_llik(theta, x)")
</hansl>
Having said this, there are several points to note: if the Hessian turns
out to be non-negative definite, it could be because you're on a saddle
point, like John said, which is unlikely; in common problems, it's far
more likely that this may be an artifact of numerical differentiation.
In the latter case, the best thing is ALWAYS to supply analytical
derivatives, but in case it's impossible, or too complicated, there are
several strategies that may help:
* try rescaling your data: numerical differentiation may struggle with
parameters that are "too large" or "too small";
* reparametrisation is also very helpful in some cases; for example, you
don't have to estimate a variance as-is. You could estimate its log, or
its square root.
* even if you don't have a function for the Hessian, the analytical
score can be a big help: compute the score and use the fdjac() function
on that.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
118
days inactive
121
days old
4 comments
4 participants
participants (4)
-
Alecos Papadopoulos -
John C Frain -
Riccardo (Jack) Lucchetti -
Sven Schreiber