As a result of some testing of my panel stochastic frontier function,
I would like to give a warning about the reliability of the test for
the accuracy of analytical derivatives. In general, it is very
useful. But the warning is that I have found that I can provoke
failures in the test simply by changing the starting values of my
parameters. I am 99.9% sure that my analytical derivatives are now
correct (though they are horribly tedious to program) because (a) I
can reproduce results generated by Stata to the 6th significant
figure using both numerical and analytical derivatives, and (b) Stata
reports identical gradients for the parameters at the same starting values.
From past experience I know that the log likelihood function for
complex stochastic frontiers is not globally concave and getting good
starting values is sometimes very difficult. Part of the reason is
that the log likelihood involves an evaluation of the cumulative
normal at what may be extreme values, which can cause degeneracy and
arbitrary fixes.
Stata frequently reports that it is in a non-concave part of the
likelihood function. What I think happens in gretl is that the
minpack numerical derivatives routine generates arc values for the
derivatives that differ from the analytical derivatives because the
slope changes abruptly. As a consequence the failure message is not
necessarily correct or helpful, because the difference may be due to
discontinuities or lack of concavity in the log-likelihood.
There is a possible solution, but there may be restrictions on
implementing it. Rather than stopping when the numerical and
analytical derivatives appear to differ, why not continue but by
using the numerical derivatives - reporting that fact. Then, every 5
or 10 iterations test whether they still differ. If the derivatives
appear to differ, continue with the numerical derivatives whereas if
they are now identical the program should switch to use the
analytical derivatives.
Gordon
Show replies by date