10:31 p.m.
Hi,
with a recent snapshot I'm seeing weird behavior when playing around
with the relatively new auto-omit option through BIC. Sorry, I cannot
provide a self-contained example with public data yet. This is with
7-days-daily data.
First, what I'm seeing is that with --auto=BIC the constant term is
eliminated. It doesn't do that with --auto=0.05 and in this data it
doesn't seem to make sense. I thought that the constant is always kept,
at least per default?
Along the elimination sequence I'm seeing: " Drop const BIC = 1.#QNAN"
... this already looks fishy, something going wrong here? There are more
lines with the BIC = 1.#QNAN value.
And then at the end gretl tells me:
"Test statistic: F(30, 128) = 157.789, p-value 5.72395e-087
Omitting variables improved 0 of 3 information criteria."
Needless to say, the resulting model is very dubious.
Maybe this information already helps a little to diagnose the problem.
thanks
sven
12:43 p.m.
On Thu, Jul 24, 2025 at 4:31 PM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
with a recent snapshot I'm seeing weird behavior when playing around
with the relatively new auto-omit option through BIC. Sorry, I cannot
provide a self-contained example with public data yet. This is with
7-days-daily data.
First, what I'm seeing is that with --auto=BIC the constant term is
eliminated. It doesn't do that with --auto=0.05 and in this data it
doesn't seem to make sense. I thought that the constant is always kept,
at least per default?
Along the elimination sequence I'm seeing: " Drop const BIC = 1.#QNAN"
... this already looks fishy, something going wrong here? There are more
lines with the BIC = 1.#QNAN value.
And then at the end gretl tells me:
"Test statistic: F(30, 128) = 157.789, p-value 5.72395e-087
Omitting variables improved 0 of 3 information criteria."
Needless to say, the resulting model is very dubious.
Maybe this information already helps a little to diagnose the problem.
Not much, I'm afraid. If you can't share the data, please at least
show the specification and output from the original regression, and
the full output from "omit". Thanks.
Allin
1:03 p.m.
Am 25.07.2025 um 12:43 schrieb Cottrell, Allin:
On Thu, Jul 24, 2025 at 4:31 PM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
> doesn't seem to make sense. I thought that the constant is always kept,
> at least per default?
>
> ...
>
> Maybe this information already helps a little to diagnose the problem.
Not much, I'm afraid. If you can't share the data, please at least
show the specification and output from the original regression, and
the full output from "omit". Thanks.
Will do -- but in the meantime, is my assumption correct that the
constant should not automatically be removed by the algorithm?
thanks
sven
1:08 p.m.
On Fri, Jul 25, 2025 at 7:03 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
Am 25.07.2025 um 12:43 schrieb Cottrell, Allin:
> On Thu, Jul 24, 2025 at 4:31 PM Sven Schreiber
> <sven.schreiber(a)fu-berlin.de> wrote:
>> doesn't seem to make sense. I thought that the constant is always kept,
>> at least per default?
>>
>> ...
>>
>> Maybe this information already helps a little to diagnose the problem.
> Not much, I'm afraid. If you can't share the data, please at least
> show the specification and output from the original regression, and
> the full output from "omit". Thanks.
Will do -- but in the meantime, is my assumption correct that the
constant should not automatically be removed by the algorithm?
Yes, that's right.
Allin
3:53 p.m.
Am 25.07.2025 um 13:08 schrieb Cottrell, Allin:
On Fri, Jul 25, 2025 at 7:03 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
> Will do -- but in the meantime, is my assumption correct that the
> constant should not automatically be removed by the algorithm?
Yes, that's right.
I'm looking at some related code in git, leading to some questions
(although not directly related to the reported failure yet).
In lib/src/compare.c there's auto_drop_var around line 1511. In that
function, it seems that a first loop picks the candidate variable to be
dropped, and always by looking at the minimal t-stat. (if (tstat < tmin)
and so on.) Only _afterwards_ does the choice of using an info criterion
come into play. (Below, where: "... else { /* using an info criterion */
...")
I can understand why one wants to focus on a candidate variable in each
round, but doesn't that contradict the documentation of omit --auto:
'The “best” candidate at each step is then that whose omission gives the
greatest improvement (reduction) in the selected criterion.' ??
Maybe I'm not looking at the really relevant path of the code, though.
thanks
sven
4:21 p.m.
Am 25.07.2025 um 15:53 schrieb Sven Schreiber:
Am 25.07.2025 um 13:08 schrieb Cottrell, Allin:
> On Fri, Jul 25, 2025 at 7:03 AM Sven Schreiber
> <sven.schreiber(a)fu-berlin.de> wrote:
>
>> Will do -- but in the meantime, is my assumption correct that the
>> constant should not automatically be removed by the algorithm?
> Yes, that's right.
In lib/src/compare.c there's auto_drop_var ...
Maybe I'm not looking at the really relevant path of the code, though.
Ah, OK, from the output (included at the end) I got the clue that
actually the used code is in plugin/stepwise.c instead. (I had discarded
that possibility because that file at the top has the comment "*Forward*
stepwise plugin for gretl,")
Anyway, so below there's an example output.
cheers
sven
------------------
Modell 2: KQ, benutze die Beobachtungen 2025-01-21:2025-06-23 (T = 154)
Abhängige Variable: rad
Koeffizient Std.-fehler t-Quotient p-Wert
----------------------------------------------------------------------
const 5536,13 56518,1 0,09795 0,9222
tempinnen 11,0701 17,2132 0,6431 0,5222
feuchtinnen 35,3635 20,3717 1,736 0,0869 *
luftdruck −12,6372 121,132 −0,1043 0,9172
feuchtaussen 1,08271 2,79138 0,3879 0,6993
tempaussen −0,900903 3,37365 −0,2670 0,7902
wind −0,602844 20,8925 −0,02885 0,9771
rad_1 0,553513 0,120039 4,611 1,72e-05 ***
rad_2 −0,159464 0,130568 −1,221 0,2260
rad_3 0,0858843 0,127545 0,6734 0,5029
rad_4 −0,130093 0,126318 −1,030 0,3066
rad_5 0,106804 0,130994 0,8153 0,4176
rad_6 −0,0969717 0,123730 −0,7837 0,4358
rad_7 −0,115546 0,115943 −0,9966 0,3224
rad_8 0,105031 0,113759 0,9233 0,3590
rad_9 −0,0587132 0,112264 −0,5230 0,6026
rad_10 −0,0761842 0,0994576 −0,7660 0,4462
tempinnen_1 7,37232 10,3823 0,7101 0,4800
tempinnen_2 −18,2027 10,7078 −1,700 0,0935 *
tempinnen_3 19,8350 11,0345 1,798 0,0765 *
tempinnen_4 −22,2215 11,0326 −2,014 0,0478 **
tempinnen_5 32,5914 10,5928 3,077 0,0030 ***
tempinnen_6 −21,0943 11,1170 −1,897 0,0618 *
tempinnen_7 18,1981 11,2022 1,625 0,1087
tempinnen_8 −9,71073 10,8887 −0,8918 0,3755
tempinnen_9 −6,67112 9,95956 −0,6698 0,5051
tempinnen_10 7,01160 5,50799 1,273 0,2072
feuchtinnen_1 4,72647 5,89112 0,8023 0,4251
feuchtinnen_2 −13,4659 5,95246 −2,262 0,0267 **
feuchtinnen_3 0,338754 6,08078 0,05571 0,9557
feuchtinnen_4 −8,71127 6,26676 −1,390 0,1688
feuchtinnen_5 19,5092 6,00384 3,249 0,0018 ***
feuchtinnen_6 −4,20423 6,21161 −0,6768 0,5007
feuchtinnen_7 7,78847 6,33687 1,229 0,2231
feuchtinnen_8 −5,11675 6,18001 −0,8280 0,4105
feuchtinnen_9 −6,78265 5,34188 −1,270 0,2083
feuchtinnen_10 0,853010 3,50815 0,2432 0,8086
luftdruck_1 3,72545 1,84353 2,021 0,0471 **
luftdruck_2 −3,41640 1,75951 −1,942 0,0561 *
luftdruck_3 0,149458 1,74713 0,08555 0,9321
luftdruck_4 0,384889 1,71969 0,2238 0,8235
luftdruck_5 0,269839 1,69044 0,1596 0,8736
luftdruck_6 −0,396276 1,59330 −0,2487 0,8043
luftdruck_7 0,353557 1,59143 0,2222 0,8248
luftdruck_8 −1,93024 1,59538 −1,210 0,2303
luftdruck_9 3,26479 1,55495 2,100 0,0393 **
luftdruck_10 −2,09401 1,23271 −1,699 0,0938 *
feuchtaussen_1 0,116539 0,623752 0,1868 0,8523
feuchtaussen_2 0,695507 0,614810 1,131 0,2618
feuchtaussen_3 −0,0543231 0,629564 −0,08629 0,9315
feuchtaussen_4 0,770473 0,641842 1,200 0,2340
feuchtaussen_5 −0,263227 0,670686 −0,3925 0,6959
feuchtaussen_6 −0,565885 0,631208 −0,8965 0,3730
feuchtaussen_7 −0,551022 0,627969 −0,8775 0,3832
feuchtaussen_8 −0,205142 0,601818 −0,3409 0,7342
feuchtaussen_9 −0,0371180 0,567678 −0,06539 0,9481
feuchtaussen_10 0,663247 0,429088 1,546 0,1266
tempaussen_1 −6,27551 3,80465 −1,649 0,1035
tempaussen_2 5,39460 4,05504 1,330 0,1877
tempaussen_3 −0,593295 4,18907 −0,1416 0,8878
tempaussen_4 3,36561 4,20894 0,7996 0,4266
tempaussen_5 0,931540 4,27281 0,2180 0,8280
tempaussen_6 −10,9364 3,94519 −2,772 0,0071 ***
tempaussen_7 5,00474 4,20909 1,189 0,2384
tempaussen_8 −10,6409 4,06967 −2,615 0,0109 **
tempaussen_9 7,27809 4,07750 1,785 0,0785 *
tempaussen_10 1,20855 2,74192 0,4408 0,6607
wind_1 −13,9894 5,19899 −2,691 0,0089 ***
wind_2 7,83067 5,59587 1,399 0,1661
wind_3 7,01675 5,51955 1,271 0,2078
wind_4 0,961869 5,37788 0,1789 0,8586
wind_5 −2,64880 5,06375 −0,5231 0,6025
wind_6 −6,70252 5,16057 −1,299 0,1982
wind_7 7,05984 4,91908 1,435 0,1556
wind_8 −5,93226 5,24085 −1,132 0,2615
wind_9 3,63178 5,07193 0,7161 0,4763
wind_10 2,11044 4,80443 0,4393 0,6618
sq_tempinnen −0,417943 0,509068 −0,8210 0,4144
sq_feuchtinnen −0,269329 0,166823 −1,614 0,1109
sq_luftdruck_aaa 0,00603205 0,0642075 0,09395 0,9254
sq_feuchtaussen_~ −0,0171198 0,0187596 −0,9126 0,3645
sq_tempaussen_aaa −0,272658 0,138516 −1,968 0,0529 *
sq_wind_aaa −0,0839240 2,96821 −0,02827 0,9775
Mittel abhängige Var. 298,0341 Stdabw. abhängige Var. 66,62345
Summe quad. Residuen 83443,39 Stdfehler Regression 34,28205
R-Quadrat 0,877130 Korrigiertes R-Quadrat 0,735224
F(82, 71) 6,181049 P-Wert(F) 1,26e-13
Log-Likelihood −703,2293 Akaike-Kriterium 1572,459
Schwarz-Kriterium 1824,526 Hannan-Quinn-Kriterium 1674,848
rho −0,171468 Durbins h NA
Abgesehen von Konstante war p-Wert am höchsten für Variable 170
(sq_wind_aaa)
? myradon <- omit --auto=BIC #0.1
Sequential elimination using BIC
Baseline BIC = 1824,53
Drop sq_wind_aaa BIC = 1819,49
Drop feuchtinnen_3 BIC = 1,#QNAN
Drop const BIC = 1,#QNAN
Drop rad_1 BIC = 1,#QNAN
Drop feuchtinnen_5 BIC = 1,#QNAN
Drop tempinnen_5 BIC = 1,#QNAN
Drop tempaussen_6 BIC = 1,#QNAN
Drop wind_1 BIC = 1,#QNAN
Drop tempaussen_8 BIC = 1,#QNAN
Drop feuchtinnen_2 BIC = 1,#QNAN
Drop luftdruck_9 BIC = 1,#QNAN
Drop luftdruck_1 BIC = 1,#QNAN
Drop tempinnen_4 BIC = 1,#QNAN
Drop sq_tempaussen_aaa BIC = 1,#QNAN
Drop luftdruck_2 BIC = 1,#QNAN
Drop tempinnen_6 BIC = 1,#QNAN
Drop tempinnen_3 BIC = 1,#QNAN
Drop tempaussen_9 BIC = 1,#QNAN
Drop feuchtinnen BIC = 1,#QNAN
Drop tempinnen_2 BIC = 1,#QNAN
Drop luftdruck_10 BIC = 1,#QNAN
Drop tempaussen_1 BIC = 1,#QNAN
Drop tempinnen_7 BIC = 1,#QNAN
Drop sq_feuchtinnen BIC = 1,#QNAN
Drop feuchtaussen_10 BIC = 1,#QNAN
Drop wind_7 BIC = 1,#QNAN
Drop wind_2 BIC = 1,#QNAN
Drop feuchtinnen_4 BIC = 1,#QNAN
Drop tempaussen_2 BIC = 1,#QNAN
Drop wind_6 BIC = 1,#QNAN
Drop tempinnen_10 BIC = 1,#QNAN
Drop wind_3 BIC = 1,#QNAN
Drop feuchtinnen_9 BIC = 1,#QNAN
Drop feuchtinnen_7 BIC = 1,#QNAN
Drop rad_2 BIC = 1,#QNAN
Drop luftdruck_8 BIC = 1,#QNAN
Drop feuchtaussen_4 BIC = 1,#QNAN
Drop tempaussen_7 BIC = 1,#QNAN
Drop wind_8 BIC = 1,#QNAN
Drop feuchtaussen_2 BIC = 1,#QNAN
Drop rad_4 BIC = 1,#QNAN
Drop rad_7 BIC = 1,#QNAN
Drop rad_8 BIC = 1,#QNAN
Drop sq_feuchtaussen_aaa BIC = 1,#QNAN
Drop feuchtaussen_6 BIC = 1,#QNAN
Drop tempinnen_8 BIC = 1,#QNAN
Drop feuchtaussen_7 BIC = 1,#QNAN
Drop feuchtinnen_8 BIC = 1,#QNAN
Drop sq_tempinnen BIC = 1,#QNAN
Drop rad_5 BIC = 1,#QNAN
Drop feuchtinnen_1 BIC = 1,#QNAN
Drop tempaussen_4 BIC = 1,#QNAN
Drop rad_6 BIC = 1,#QNAN
Drop rad_10 BIC = 1,#QNAN
Drop wind_9 BIC = 1,#QNAN
Drop tempinnen_1 BIC = 1,#QNAN
Drop feuchtinnen_6 BIC = 1,#QNAN
Drop rad_3 BIC = 1,#QNAN
Drop tempinnen_9 BIC = 1,#QNAN
Drop tempinnen BIC = 1,#QNAN
Drop wind_5 BIC = 1,#QNAN
Drop rad_9 BIC = 1,#QNAN
Drop tempaussen_10 BIC = 1,#QNAN
Drop wind_10 BIC = 1,#QNAN
Drop feuchtaussen_5 BIC = 1,#QNAN
Drop feuchtaussen BIC = 1,#QNAN
Drop feuchtaussen_8 BIC = 1,#QNAN
Drop tempaussen BIC = 1,#QNAN
Drop luftdruck_6 BIC = 1,#QNAN
Drop feuchtinnen_10 BIC = 1,#QNAN
Drop luftdruck_4 BIC = 1,#QNAN
Drop luftdruck_7 BIC = 1,#QNAN
Drop tempaussen_5 BIC = 1,#QNAN
Drop feuchtaussen_1 BIC = 1,#QNAN
Drop wind_4 BIC = 1,#QNAN
Drop luftdruck_5 BIC = 1,#QNAN
Drop tempaussen_3 BIC = 1,#QNAN
Drop luftdruck BIC = 1,#QNAN
Drop sq_luftdruck_aaa BIC = 1,#QNAN
Drop feuchtaussen_3 BIC = 1,#QNAN
Drop luftdruck_3 BIC = 1,#QNAN
Drop feuchtaussen_9 BIC = 1,#QNAN
Test von Modell 2:
Nullhypothese: Die Regressionskoeffizienten sind Null für die Variablen
const, tempinnen, feuchtinnen, luftdruck, feuchtaussen, tempaussen,
rad_1, rad_2, rad_3, rad_4, rad_5, rad_6, rad_7, rad_8, rad_9,
rad_10, tempinnen_1, tempinnen_2, tempinnen_3, tempinnen_4,
tempinnen_5,
tempinnen_6, tempinnen_7, tempinnen_8, tempinnen_9, tempinnen_10,
feuchtinnen_1, feuchtinnen_2, feuchtinnen_3, feuchtinnen_4,
feuchtinnen_5,
feuchtinnen_6, feuchtinnen_7, feuchtinnen_8, feuchtinnen_9,
feuchtinnen_10,
luftdruck_1, luftdruck_2, luftdruck_3, luftdruck_4, luftdruck_5,
luftdruck_6, luftdruck_7, luftdruck_8, luftdruck_9, luftdruck_10,
feuchtaussen_1, feuchtaussen_2, feuchtaussen_3, feuchtaussen_4,
feuchtaussen_5, feuchtaussen_6, feuchtaussen_7, feuchtaussen_8,
feuchtaussen_9, feuchtaussen_10, tempaussen_1, tempaussen_2,
tempaussen_3, tempaussen_4, tempaussen_5, tempaussen_6, tempaussen_7,
tempaussen_8, tempaussen_9, tempaussen_10, wind_1, wind_2, wind_3,
wind_4, wind_5, wind_6, wind_7, wind_8, wind_9, wind_10, sq_tempinnen,
sq_feuchtinnen, sq_luftdruck_aaa, sq_feuchtaussen_aaa,
sq_tempaussen_aaa,
sq_wind_aaa
Teststatistik: F(82, 71) = 18,9514, p-Wert 4,09444e-028
Das Weglassen von Variablen verbesserte 0 von 3 Informationskriterien.
myradon: KQ, benutze die Beobachtungen 2025-01-11:2025-06-23 (T = 164)
Abhängige Variable: rad
Koeffizient Std.-fehler t-Quotient p-Wert
-------------------------------------------------------------
wind 88,6342 2,90040 30,56 2,24e-069 ***
Mittel abhängige Var. 299,7421
Stdabw. abhängige Var. 66,60399
Summe quad. Residuen 2297082
Stdfehler Regression 118,7119
Unzentriertes R-Quadrat 0,851396
Zentriertes R-Quadrat -2,176789
F(1, 163) 933,8735
P-Wert(F) 2,24e-69
Log-Likelihood −1015,583
Akaike-Kriterium 2033,166
Schwarz-Kriterium 2036,266
Hannan-Quinn-Kriterium 2034,425
rho 0,548149
Durbin-Watson-Stat 0,862502
4:48 p.m.
On Fri, Jul 25, 2025 at 9:53 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
I'm looking at some related code in git, leading to some questions
(although not directly related to the reported failure yet).
In lib/src/compare.c there's auto_drop_var around line 1511. In that
function, it seems that a first loop picks the candidate variable to be
dropped, and always by looking at the minimal t-stat. (if (tstat < tmin)
and so on.) Only _afterwards_ does the choice of using an info criterion
come into play. (Below, where: "... else { /* using an info criterion */
...")
I can understand why one wants to focus on a candidate variable in each
round, but doesn't that contradict the documentation of omit --auto:
'The “best” candidate at each step is then that whose omission gives the
greatest improvement (reduction) in the selected criterion.' ??
BIC = -2 * loglik + k * log(n)
The sample size, n, is a constant in context, and dropping any
regressor will reduce k by 1. All that matters for the comparison of
candidates is the least squares loglikelihood, and the SSR is only
variable in that calculation. Omitting the regressor with the smallest
absolute t-ratio will produce the smallest possible increase in the
SSR, and hence the best post-omission BIC.
Allin
4:57 p.m.
Am 25.07.2025 um 16:48 schrieb Cottrell, Allin:
On Fri, Jul 25, 2025 at 9:53 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
> I can understand why one wants to focus on a candidate variable in each
> round, but doesn't that contradict the documentation of omit --auto:
> 'The “best” candidate at each step is then that whose omission gives the
> greatest improvement (reduction) in the selected criterion.' ??
BIC = -2 * loglik + k * log(n)
The sample size, n, is a constant in context, and dropping any
regressor will reduce k by 1. All that matters for the comparison of
candidates is the least squares loglikelihood, and the SSR is only
variable in that calculation. Omitting the regressor with the smallest
absolute t-ratio will produce the smallest possible increase in the
SSR, and hence the best post-omission BIC.
Ha, excellent point!
-s
5:28 p.m.
On Fri, Jul 25, 2025 at 10:22 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
Am 25.07.2025 um 15:53 schrieb Sven Schreiber:
> Am 25.07.2025 um 13:08 schrieb Cottrell, Allin:
>> On Fri, Jul 25, 2025 at 7:03 AM Sven Schreiber
>> <sven.schreiber(a)fu-berlin.de> wrote:
>>
>>> Will do -- but in the meantime, is my assumption correct that the
>>> constant should not automatically be removed by the algorithm?
>> Yes, that's right.
>
> In lib/src/compare.c there's auto_drop_var ...
>
> Maybe I'm not looking at the really relevant path of the code, though.
>
Ah, OK, from the output (included at the end) I got the clue that
actually the used code is in plugin/stepwise.c instead. (I had discarded
that possibility because that file at the top has the comment "*Forward*
stepwise plugin for gretl,")
Anyway, so below there's an example output [...]
Thanks, that helps. I don't know why BIC is becoming NaN, but it's
clear that we need to flag an error and break out of the omission loop
with an informative message as soon as that happens. That's now
arranged in git and I should be able to put up new snapshots within
the next couple of hours.
Allin
7:07 p.m.
Am 25.07.2025 um 17:28 schrieb Cottrell, Allin:
On Fri, Jul 25, 2025 at 10:22 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
> Ah, OK, from the output (included at the end) I got the clue that
> actually the used code is in plugin/stepwise.c instead. (I had discarded
> that possibility because that file at the top has the comment "*Forward*
> stepwise plugin for gretl,")
>
> Anyway, so below there's an example output [...]
>
Thanks, that helps. I don't know why BIC is becoming NaN, but it's
clear that we need to flag an error and break out of the omission loop
with an informative message as soon as that happens. That's now
arranged in git and I should be able to put up new snapshots within
the next couple of hours.
Thanks, Allin. However, I think we need to get at the root of the
problem as well. I don't see any good (=numerically realistic) reason
why BIC couldn't be computed where the pvalue could, right? So something
with the internal handling still must go wrong I believe.
cheers
sven
7:27 p.m.
On Fri, 25 Jul 2025, Sven Schreiber wrote:
Am 25.07.2025 um 17:28 schrieb Cottrell, Allin:
> On Fri, Jul 25, 2025 at 10:22 AM Sven Schreiber
> <sven.schreiber(a)fu-berlin.de> wrote:
>> Ah, OK, from the output (included at the end) I got the clue that
>> actually the used code is in plugin/stepwise.c instead. (I had discarded
>> that possibility because that file at the top has the comment "*Forward*
>> stepwise plugin for gretl,")
>>
>> Anyway, so below there's an example output [...]
>>
> Thanks, that helps. I don't know why BIC is becoming NaN, but it's
> clear that we need to flag an error and break out of the omission loop
> with an informative message as soon as that happens. That's now
> arranged in git and I should be able to put up new snapshots within
> the next couple of hours.
Thanks, Allin. However, I think we need to get at the root of the
problem as well.
Yes, of course.
I don't see any good (=numerically realistic) reason why BIC
couldn't be computed where the pvalue could, right? So something
with the internal handling still must go wrong I believe.
It's something to do with lags. With time-series data and a
specification that includes lags (like yours), SSR goes NaN and
therefore BIC goes NaN when a regressor that's a lag is omitted.
None of our test cases to date were of this sort, so it's just as
well that you tried your experiment. I'm working on trying to
understand exactly how the bad SSR arises.
Allin
9:11 p.m.
Am 25.07.2025 um 19:27 schrieb Allin Cottrell:
It's something to do with lags. With time-series data and a
specification that includes lags (like yours), SSR goes NaN and
therefore BIC goes NaN when a regressor that's a lag is omitted.
I think it's more subtle - I have variations here where some lags are
omitted and BIC is OK, and then later the problem happens.
No other insight so far.
-s
9:30 p.m.
On Fri, Jul 25, 2025 at 3:11 PM Sven Schreiber <sven.schreiber(a)fu-berlin.de>
wrote:
Am 25.07.2025 um 19:27 schrieb Allin Cottrell:
>
> It's something to do with lags. With time-series data and a
> specification that includes lags (like yours), SSR goes NaN and
> therefore BIC goes NaN when a regressor that's a lag is omitted.
I think it's more subtle - I have variations here where some lags are
omitted and BIC is OK, and then later the problem happens.
It's more specific than I indicated: things go wrong as soon as a lagged
regressor *that is not the last regressor at the time*, gets omitted.
The regressors will have been reordered, if necessary, at the start of the
backward stepwise procedure, to increase the likelihood that terms to be
omitted will come at the end, which simplifies things considerably. But
that's not going to work 100%. When we have to eliminate an "internal"
column of Q, and row/column of R, the recomputation that's needed involves
reference to the dataset, but at what starting point? We were diving in at
dset->t1, but in a specification with lags it should be at orig->t1,
where @orig is the original model. That's a distinction without a
difference in the case of cross-sectional data, but it's crucial for time
series with lags. Now that's fixed in git.
Allin
10:33 p.m.
Am 25.07.2025 um 21:30 schrieb Cottrell, Allin:
It's more specific than I indicated: things go wrong as soon as a
lagged regressor /that is not the last regressor at the time/, gets
omitted.
The regressors will have been reordered, if necessary, at the start of
the backward stepwise procedure, to increase the likelihood that terms
to be omitted will come at the end, which simplifies things
considerably. But that's not going to work 100%. When we have to
eliminate an "internal" column of Q, and row/column of R, the
recomputation that's needed involves reference to the dataset, but at
what starting point? We were diving in at dset->t1, but in a
specification with lags it should be at orig->t1, where @orig is the
original model. That's a distinction without a difference in the case
of cross-sectional data, but it's crucial for time series with
lags.Now that's fixed in git.
Thanks, Allin! I tested with the latest snapshot and everything seems
fine now!
cheers
sven
2:23 p.m.
On Sun, 27 Jul 2025, Sven Schreiber wrote:
Am 25.07.2025 um 21:30 schrieb Cottrell, Allin:
> It's more specific than I indicated: things go wrong as soon as a lagged
> regressor /that is not the last regressor at the time/, gets omitted.
>
> The regressors will have been reordered, if necessary, at the start of the
> backward stepwise procedure, to increase the likelihood that terms to be
> omitted will come at the end, which simplifies things considerably. But
> that's not going to work 100%. When we have to eliminate an "internal"
> column of Q, and row/column of R, the recomputation that's needed involves
> reference to the dataset, but at what starting point? We were diving in at
> dset->t1, but in a specification with lags it should be at orig->t1,
> where @orig is the original model. That's a distinction without a
> difference in the case of cross-sectional data, but it's crucial for time
> series with lags.Now that's fixed in git.
Thanks, Allin! I tested with the latest snapshot and everything seems fine
now!
Ah, glad to hear it.
I've now committed a second round of corrections for add/omit with
the --auto flag, this time to implement proper handling of missing
values.
There's one case that provokes a "missing data" error. That's with
"add", when one or more of the candidate series for addition include
missing values within the original estimation range that are not
aligned with missing values in the dependent variable or the
original regressors. We refuse that case since it would force a
reduction of the sample relative to the original estimation. A
simple example follows.
<hansl>
open data4-10
ENROLL[30] = NA # this is alright now
MEMNEA[30] = NA # also OK now
# MEMNEA[40] = NA # provokes an error
list X = 2..9
ols ENROLL const
add X --auto=BIC
</hansl>
Allin
3:34 p.m.
Am 28.07.2025 um 14:23 schrieb Allin Cottrell:
On Sun, 27 Jul 2025, Sven Schreiber wrote:
>
> Thanks, Allin! I tested with the latest snapshot and everything seems
> fine now!
Ah, glad to hear it.
I've now committed a second round of corrections for add/omit with the
--auto flag, this time to implement proper handling of missing values.
There's one case that provokes a "missing data" error. That's with
"add", when one or more of the candidate series for addition include
missing values within the original estimation range that are not
aligned with missing values in the dependent variable or the original
regressors. We refuse that case since it would force a reduction of
the sample relative to the original estimation. A simple example follows.
That sounds like the right thing to do. BTW, I also think that the
original sample range should be maintained with "omit" even if the
variable responsible for the sample restriction were dropped. I believe
this is already the case (right?), but maybe we should also have a test
case for that. I can make one up if you want.
cheers
sven
8:50 p.m.
On Mon, Jul 28, 2025 at 9:34 AM Sven Schreiber
<sven.schreiber(a)fu-berlin.de> wrote:
Am 28.07.2025 um 14:23 schrieb Allin Cottrell:
>
> There's one case that provokes a "missing data" error. That's
with
> "add", when one or more of the candidate series for addition include
> missing values within the original estimation range that are not
> aligned with missing values in the dependent variable or the original
> regressors. We refuse that case since it would force a reduction of
> the sample relative to the original estimation. A simple example follows.
That sounds like the right thing to do. BTW, I also think that the
original sample range should be maintained with "omit" even if the
variable responsible for the sample restriction were dropped. I believe
this is already the case (right?), but maybe we should also have a test
case for that. I can make one up if you want.
Yes, that's right: we preserve the original sample, even when omitting
a regressor with missing values would permit a bigger sample.
Allin
14
days inactive
18
days old
16 comments
3 participants
participants (3)
-
Allin Cottrell -
Cottrell, Allin
-
Sven Schreiber