Re: [Gretl-devel] arima-Problem: unexpected $bic values
On Thu, 23 Mar 2017, Sven Schreiber wrote:
Hi,
first I've noticed a strange behavior with Jack's BNdecomp.gfn package when I
used the auto choice of AR/MA orders. But I think I've tracked this down to
what the $bic accessor gives with gretl's arima command. (So I don't think
this is a problem with BNdecomp, it just shows up there.)
Below is a complete example of what I mean. The $bic value for an
ARIMA(1,1,0) model --which is obviously just an AR(1) in first differences--
is noticeably higher when gretl's "arima" command is used, compared to what
the manual AR(1) to the diffs gives me. [...]
Thanks for the full example. There are two things going on here.
1) The convention when calculating BIC from a model estimated by least
squares is to set "k" to the number of regression coefficients
(leaving aside the error variance), while the convention under MLE is
to include the variance estimator in k. (Or at least I think that's a
fair statement of the case.) This is one way a $bic discrepancy could
arise (for the same value of the loglikelihood).
This raises a question about our "arma" command: for a pure "I" model
(p=q=0), in which case OLS = MLE under normality, should we report the
information criteria compatibly with "ols", or compatibly with arma
variants estimated via ML?
2) An ARIMA(1,1,0) model can be estimated via OLS on differenced data
but that's not equivalent to exact ML (it's equivalent to conditional
ML), meaning that you'll get a difference in the log-likelihood (and
hence the BIC). I've checked that gretl's ARIMA(1,1,0) output matches
that of X-13-ARIMA: they both differ from OLS in the same way.
Now in your comparison of two ARIMA(p,1,0) models, you got (with *
indicating "best" according to BIC):
1) Using OLS
p = 0: BIC = 1923.8
p = 1: BIC = 1922.6*
2) Using the "arma" command
p = 0: BIC = 1923.8*
p = 1: BIC = 1929.8
In light of the comments above, what's happening here? Well, at
present the BIC value shown by "arma" for p = 0 is as per OLS.
Calculating it compatibly with ML means adding ln(T), in which case
the comparison becomes:
p = 0: BIC = 1928.4*
p = 1: BIC = 1929.8
So BIC under exact ML still favors the p = 0 model, though only
marginally. That seems OK: note that the ML estimate of phi is not
significant at the 5 percent level, and the improvement in the log
likelihood relative to p = 0 is very small.
For the benefit of hansl code that uses AIC or BIC to select ARIMA
specification, I guess we should switch to ML-compatible calculation
of information criteria when we end up using least squares under
"arma".
Allin