1:21 p.m.
Hi,
prompted by Jack's recent biprobit explanation in the guide I checked
whether the automatically run test is available via $test -- it's not
(gives NA).
I know biprobit is not a test but an estimation command. Does that mean
that $test/$pvalue shouldn't be used there? Should there be a new
'modtest' incarnation?
thanks
sven
1:36 p.m.
On Tue, 27 Oct 2020, Sven Schreiber wrote:
Hi,
prompted by Jack's recent biprobit explanation in the guide I checked
whether the automatically run test is available via $test -- it's not
(gives NA).
I know biprobit is not a test but an estimation command. Does that mean
that $test/$pvalue shouldn't be used there? Should there be a new
'modtest' incarnation?
The test is available as $model.independence_test. This should be
mentioned in the Guide. There are few typos in that area and I can
make the changes later today.
Allin
1:44 p.m.
Am 27.10.2020 um 14:36 schrieb Allin Cottrell:
On Tue, 27 Oct 2020, Sven Schreiber wrote:
> Hi,
>
> prompted by Jack's recent biprobit explanation in the guide I checked
> whether the automatically run test is available via $test -- it's not
> (gives NA).
>
> I know biprobit is not a test but an estimation command. Does that mean
> that $test/$pvalue shouldn't be used there? Should there be a new
> 'modtest' incarnation?
The test is available as $model.independence_test. This should be
mentioned in the Guide. There are few typos in that area and I can
make the changes later today.
Excellent, thanks.
Somewhat related: The printed test result is the same whether or not the
--robust option is chosen for the biprobit command. Now I see that the
test is LR and as such of course doesn't use the VCVs. But is it
actually valid for example under heteroskedastic errors? If not, perhaps
it shouldn't be carried out then.
thanks
svne
5:38 p.m.
Am 27.10.2020 um 14:44 schrieb Sven Schreiber:
Somewhat related: The printed test result is the same whether or not the
--robust option is chosen for the biprobit command. Now I see that the
test is LR and as such of course doesn't use the VCVs. But is it
actually valid for example under heteroskedastic errors? If not, perhaps
it shouldn't be carried out then.
ping - somebody have an opinion about this?
thanks
sven
9:47 p.m.
On Fri, 30 Oct 2020, Sven Schreiber wrote:
Am 27.10.2020 um 14:44 schrieb Sven Schreiber:
>
> Somewhat related: The printed test result is the same whether or not the
> --robust option is chosen for the biprobit command. Now I see that the
> test is LR and as such of course doesn't use the VCVs. But is it
> actually valid for example under heteroskedastic errors? If not, perhaps
> it shouldn't be carried out then.
>
ping - somebody have an opinion about this?
The LR test is the traditional way of checking the independence
hypothesis, but I have no idea if it's statistically tenable in the
context of QML estimation. On the other hand, its is relatively simple to
set up a (pseudo) Wald test for the same hypothesis. I'll have a look at
this.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:41 p.m.
On Fri, 30 Oct 2020, Riccardo (Jack) Lucchetti wrote:
The LR test is the traditional way of checking the independence
hypothesis,
but I have no idea if it's statistically tenable in the context of QML
estimation. On the other hand, its is relatively simple to set up a (pseudo)
Wald test for the same hypothesis. I'll have a look at this.
Hm, this will require some more work than I had thought. In fact, I'm not
quite sure we're doing the right thing in the current incarnation of
biprobit. Let me explain. The number of estimated coefficients is k1 + k2
+ 1, where k1 and k2 are the numbers of explanatory variables in the first
and second equation respectively, plus there's rho (actually, its
hypherbolic arctangent).
When we record $vcv into the MODEL struct, we treat the model as if the
number of parameters were k1 + k2, although the covariance matrix is 1
element bigger, with the result that
(a) the $vcv you now get after biprobit is buggy (it's not symmetric)
(b) we don't have in our MODEL struct the estimated variance for
atan(rho).
Note that we treat $coeff in the same way, that is we don't store rho
among the estimated parameters, although its estimate is accessible as
$rho or $model.rho.
So I'm wondering: while we're at it, should we "correct" this, at the
risk
of backward incompatibility, by including rho in $coeff and making $vcv a
square matrix of dimension (k1 + k2 + 1), or shall we just fix $vcv and
maybe provide an accessor for the standard error of rho?
IMO we should change the output of biprobit and make rho a "first class
citizen", that is include it in the output just like the other estimated
parameters as well as storing it as the last element of $coeff and
enlarging $vcv and $stderr as needed, but I'd like to hear your opinion on
this before I start working on the code.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:02 a.m.
Am 30.10.2020 um 23:41 schrieb Riccardo (Jack) Lucchetti:
Hm, this will require some more work than I had thought. In fact,
I'm
not quite sure we're doing the right thing in the current incarnation of
biprobit. Let me explain. The number of estimated coefficients is k1 +
k2 + 1, where k1 and k2 are the numbers of explanatory variables in the
first and second equation respectively, plus there's rho (actually, its
hypherbolic arctangent).
When we record $vcv into the MODEL struct, we treat the model as if the
number of parameters were k1 + k2, although the covariance matrix is 1
element bigger, with the result that
(a) the $vcv you now get after biprobit is buggy (it's not symmetric)
(b) we don't have in our MODEL struct the estimated variance for atan(rho).
Note that we treat $coeff in the same way, that is we don't store rho
among the estimated parameters, although its estimate is accessible as
$rho or $model.rho.
So I'm wondering: while we're at it, should we "correct" this, at the
risk of backward incompatibility, by including rho in $coeff and making
$vcv a square matrix of dimension (k1 + k2 + 1), or shall we just fix
$vcv and maybe provide an accessor for the standard error of rho?
If rho is appended at the end wouldn't that sort of minimize the disruption?
IMO we should change the output of biprobit and make rho a
"first class
citizen", that is include it in the output just like the other estimated
parameters as well as storing it as the last element of $coeff and
enlarging $vcv and $stderr as needed, but I'd like to hear your opinion
on this before I start working on the code.
If the previous thing was buggy, then I guess the incompatibility is
somehow unavoidable. In that sense I'd say go ahead.
cheers
sven
11:09 a.m.
Am 31.10.20 um 12:02 schrieb Sven Schreiber:
Am 30.10.2020 um 23:41 schrieb Riccardo (Jack) Lucchetti:
> IMO we should change the output of biprobit and make rho a "first class
> citizen", that is include it in the output just like the other estimated
> parameters as well as storing it as the last element of $coeff and
> enlarging $vcv and $stderr as needed, but I'd like to hear your opinion
> on this before I start working on the code.
If the previous thing was buggy, then I guess the incompatibility is
somehow unavoidable. In that sense I'd say go ahead.
I agree!
Artur
1:34 p.m.
On Sat, 31 Oct 2020, Sven Schreiber wrote:
> IMO we should change the output of biprobit and make rho a
"first class
> citizen", that is include it in the output just like the other estimated
> parameters as well as storing it as the last element of $coeff and
> enlarging $vcv and $stderr as needed, but I'd like to hear your opinion
> on this before I start working on the code.
If the previous thing was buggy, then I guess the incompatibility is
somehow unavoidable. In that sense I'd say go ahead.
Not necessarily. We may decide to keep $coeff and $vcv the way they are
now (that is, without rho) and just fix $vcv. But as I said, I'd be
happier if we changed this design decision. If Allin gives me the green
light on this, I'll try to get things done asap (not before Monday,
thoough, I'm afraid).
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:39 p.m.
On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
On Sat, 31 Oct 2020, Sven Schreiber wrote:
>> IMO we should change the output of biprobit and make rho a "first class
>> citizen", that is include it in the output just like the other estimated
>> parameters as well as storing it as the last element of $coeff and
>> enlarging $vcv and $stderr as needed, but I'd like to hear your opinion
>> on this before I start working on the code.
>
> If the previous thing was buggy, then I guess the incompatibility is
> somehow unavoidable. In that sense I'd say go ahead.
Not necessarily. We may decide to keep $coeff and $vcv the way they are now
(that is, without rho) and just fix $vcv. But as I said, I'd be happier if we
changed this design decision. If Allin gives me the green light on this, I'll
try to get things done asap (not before Monday, thoough, I'm afraid).
The "fix up $vcv" option occurred to me too, though I'm not saying
it's the best solution. Probably better to incorporate rho. BTW, in
the biprobit examples I checked I haven't seen the non-square $vcv
that Jack mentioned, though I have seen some negative diagonal
elements!
While we're at it, I think there may be one or two other cases of ML
estimation where there's an "extra" parameter excluded from $coeff,
$stderr and $vcv and tacked onto the model struct separately. But
I'll need to check properly.
Allin
4:14 p.m.
On Sat, 31 Oct 2020, Allin Cottrell wrote:
On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
> On Sat, 31 Oct 2020, Sven Schreiber wrote:
>
>>> IMO we should change the output of biprobit and make rho a "first class
>>> citizen", that is include it in the output just like the other
estimated
>>> parameters as well as storing it as the last element of $coeff and
>>> enlarging $vcv and $stderr as needed, but I'd like to hear your opinion
>>> on this before I start working on the code.
>>
>> If the previous thing was buggy, then I guess the incompatibility is
>> somehow unavoidable. In that sense I'd say go ahead.
>
> Not necessarily. We may decide to keep $coeff and $vcv the way they are now
> (that is, without rho) and just fix $vcv. But as I said, I'd be happier if
> we changed this design decision. If Allin gives me the green light on this,
> I'll try to get things done asap (not before Monday, thoough, I'm afraid).
The "fix up $vcv" option occurred to me too, though I'm not saying it's
the
best solution. Probably better to incorporate rho. BTW, in the biprobit
examples I checked I haven't seen the non-square $vcv that Jack mentioned,
though I have seen some negative diagonal elements!
Sorry, I meant "non-symmetric". That's where the negative diagonal
elements come from. They are, in fact, misplaced covariances.
While we're at it, I think there may be one or two other cases of
ML
estimation where there's an "extra" parameter excluded from $coeff, $stderr
and $vcv and tacked onto the model struct separately. But I'll need to check
properly.
Ah, good idea.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
4:35 p.m.
On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
On Sat, 31 Oct 2020, Allin Cottrell wrote:
> On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
>
>> On Sat, 31 Oct 2020, Sven Schreiber wrote:
>>
>>>> IMO we should change the output of biprobit and make rho a "first
class
>>>> citizen", that is include it in the output just like the other
estimated
>>>> parameters as well as storing it as the last element of $coeff and
>>>> enlarging $vcv and $stderr as needed, but I'd like to hear your
opinion
>>>> on this before I start working on the code.
>>>
>>> If the previous thing was buggy, then I guess the incompatibility is
>>> somehow unavoidable. In that sense I'd say go ahead.
>>
>> Not necessarily. We may decide to keep $coeff and $vcv the way they are
>> now (that is, without rho) and just fix $vcv. But as I said, I'd be
>> happier if we changed this design decision. If Allin gives me the green
>> light on this, I'll try to get things done asap (not before Monday,
>> thoough, I'm afraid).
>
> The "fix up $vcv" option occurred to me too, though I'm not saying
it's the
> best solution. Probably better to incorporate rho. BTW, in the biprobit
> examples I checked I haven't seen the non-square $vcv that Jack mentioned,
> though I have seen some negative diagonal elements!
Sorry, I meant "non-symmetric". That's where the negative diagonal elements
come from. They are, in fact, misplaced covariances.
Ah, that makes sense. One more thought: there could be strange
fallout from including rho in $coeff and $stderr when it comes to
printing the model output, and possibly other contexts too. Needs
testing anyway.
Allin
5:34 p.m.
On Sat, 31 Oct 2020, Allin Cottrell wrote:
On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
> On Sat, 31 Oct 2020, Allin Cottrell wrote:
>
>> On Sat, 31 Oct 2020, Riccardo (Jack) Lucchetti wrote:
>>
>>> On Sat, 31 Oct 2020, Sven Schreiber wrote:
>>>
>>>>> IMO we should change the output of biprobit and make rho a
"first class
>>>>> citizen", that is include it in the output just like the other
>>>>> estimated
>>>>> parameters as well as storing it as the last element of $coeff and
>>>>> enlarging $vcv and $stderr as needed, but I'd like to hear your
opinion
>>>>> on this before I start working on the code.
>>>>
>>>> If the previous thing was buggy, then I guess the incompatibility is
>>>> somehow unavoidable. In that sense I'd say go ahead.
>>>
>>> Not necessarily. We may decide to keep $coeff and $vcv the way they are
>>> now (that is, without rho) and just fix $vcv. But as I said, I'd be
>>> happier if we changed this design decision. If Allin gives me the green
>>> light on this, I'll try to get things done asap (not before Monday,
>>> thoough, I'm afraid).
>>
>> The "fix up $vcv" option occurred to me too, though I'm not saying
it's
>> the best solution. Probably better to incorporate rho. BTW, in the
>> biprobit examples I checked I haven't seen the non-square $vcv that Jack
>> mentioned, though I have seen some negative diagonal elements!
>
> Sorry, I meant "non-symmetric". That's where the negative diagonal
elements
> come from. They are, in fact, misplaced covariances.
Ah, that makes sense. One more thought: there could be strange fallout from
including rho in $coeff and $stderr when it comes to printing the model
output, and possibly other contexts too. Needs testing anyway.
Since we have a release coming up -- once we've cleared the queue of
pressing bugs! -- I've committed to git what I believe is a
relatively safe fix for the broken $vcv after biprobit, namely
excluding the row and column pertaining to the "extra" parameter
rho.
The mess-up with $vcv was due to writing a covariance matrix of
dimension k+1 into the model struct, then accessing it on the
assumption it was of dimension k. Now we trim it to k before writing
it to the model.
We can revisit this later. It may be better to promote rho to full
"coeff" status, but for now I'm concerned about possible fallout
from that move.
Allin
10:11 a.m.
On Sun, 1 Nov 2020, Allin Cottrell wrote:
Since we have a release coming up -- once we've cleared the queue
of pressing
bugs! -- I've committed to git what I believe is a relatively safe fix for
the broken $vcv after biprobit, namely excluding the row and column
pertaining to the "extra" parameter rho.
The mess-up with $vcv was due to writing a covariance matrix of dimension k+1
into the model struct, then accessing it on the assumption it was of
dimension k. Now we trim it to k before writing it to the model.
We can revisit this later. It may be better to promote rho to full "coeff"
status, but for now I'm concerned about possible fallout from that move.
Fine by me. I spent some time trying to "fix" the independence test for
the --robust case, but gave up eventually because there's too much code
we'd have to add to the biprobit plugin code and it's not really
worthwhile.
In practice, we'd have to repeat all the matrix calculations for the vcv
twice just for finding one number. This will be completely unnecessary
once we include rho into the MODEL struct, as we plan to do after release.
By the way, it's not even entirely clear to me whether the LR test is
invalid or not in the context of QMLE estimation (although I suspect it
is).
So I guess that, for now, it's enough to add a cautionary note to the
biprobit docs, and we can modify it as needed later on, when we provide an
independence test that is asymptotically valid even for QMLE estimation.
I'll do this later today.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:14 a.m.
Am 02.11.2020 um 11:11 schrieb Riccardo (Jack) Lucchetti:
On Sun, 1 Nov 2020, Allin Cottrell wrote:
release. By the way, it's not even entirely clear to me whether
the LR
test is invalid or not in the context of QMLE estimation (although I
suspect it is).
So I guess that, for now, it's enough to add a cautionary note to the
biprobit docs, and we can modify it as needed later on, when we provide
an independence test that is asymptotically valid even for QMLE
estimation. I'll do this later today.
I think it wouldn't hurt to print the cautionary note also in the
output. (For the robust case I mean.)
And in general I think a tracker ticket should be created for this so it
won't be forgotten.
cheers
sven
12:10 p.m.
On Mon, 2 Nov 2020, Sven Schreiber wrote:
Am 02.11.2020 um 11:11 schrieb Riccardo (Jack) Lucchetti:
> On Sun, 1 Nov 2020, Allin Cottrell wrote:
> release. By the way, it's not even entirely clear to me whether the LR
> test is invalid or not in the context of QMLE estimation (although I
> suspect it is).
>
> So I guess that, for now, it's enough to add a cautionary note to the
> biprobit docs, and we can modify it as needed later on, when we provide
> an independence test that is asymptotically valid even for QMLE
> estimation. I'll do this later today.
Just done.
I think it wouldn't hurt to print the cautionary note also in
the
output. (For the robust case I mean.)
It wouldn't hurt, but I didn't do this since I couldn't be bothered with
adding another translatable string to the po files for something we're
going to fix soon anyway.
And in general I think a tracker ticket should be created for this so
it
won't be forgotten.
Good idea. Done.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
1264
days inactive
1270
days old
15 comments
4 participants
participants (4)
-
Allin Cottrell -
Artur Tarassow -
Riccardo (Jack) Lucchetti -
Sven Schreiber