On Sat, 17 Nov 2012, Lee Adkins wrote: > I've encountered the following "feature." restrict is putting NA into the > standard error rather than 0, making it impossible to retrieve the > estimated standard errors as a matrix (or anything else). Is this as > intended?

Here, the computed variance of the restricted estimator has a negative diagonal entry, so you get a NaN among the standard errors. However, maybe we should return the resulting matrix as $stderr, NaNs and all:

On 11/18/2012 03:43 AM, Allin Cottrell wrote: > On Sat, 17 Nov 2012, Lee Adkins wrote: > > You're estimating the model subject to a restriction that is > violently at odds with the data (F(6, 1073) = 463) and you're > stressing the numerical apparatus to breaking point. > > For comparison, here's the estimation done manually using William > Greene's formulae: > ... > Here, the computed variance of the restricted estimator has a > negative diagonal entry, so you get a NaN among the standard errors. This is a great opportunity to repeat what I wrote in the slightly different context of the Hausman test (http://econ.schreiberlin.de/schreiberresearch.html#hausman): If you get such weirdness such as negative estimated variances, chances are that the restriction should be rejected, which is exactly the opposite of what Greene and others have been recommending (by setting the test statistic to zero). With respect to gretl's behavior: Some warning message as used to be the case with the Hausman test would be nice I guess. What I didn't understand from Lee's output is why there were values displayed for the constant term if everything else fails.

On Sun, 18 Nov 2012, Allin Cottrell wrote: > If we were to do this, I'd favour restricting the "clean-up" to the > standard errors (printing 0 rather than NA) and let the $vcv > accessor show what was actually computed, warts and all. I think that the flaws from machine precision are of great didactical value. IMHO, teaching students that 1.2345e-30 is in fact zero and they should _distrust_ software that writes "0" instead of 1.2345e-30 is part of teaching good econometrics. That said, in a case like the one Lee brought up, better to have 0 than NA.

On Sun, Nov 18, 2012 at 8:51 AM, Allin Cottrell <cottrell(a)wfu.edu&gt; wrote: > On Sun, 18 Nov 2012, Riccardo (Jack) Lucchetti wrote: > > > On Sun, 18 Nov 2012, Allin Cottrell wrote: > > > >> If we were to do this, I'd favour restricting the "clean-up" to the > >> standard errors (printing 0 rather than NA) and let the $vcv > >> accessor show what was actually computed, warts and all. > > > > I think that the flaws from machine precision are of great didactical > value. > > IMHO, teaching students that 1.2345e-30 is in fact zero and they should > > _distrust_ software that writes "0" instead of 1.2345e-30 is part of > teaching > > good econometrics. That said, in a case like the one Lee brought up, > better > > to have 0 than NA. > > Agreed. > > There are two aspects of our policy to date that are > questionable. First, when computing standard errors from a > variance matrix we've always set the s.e. to NA when we > encounter a negative diagonal entry. This is reasonable in > general, but is arguably too strict when we're producing > restricted estimates. > > When we estimate subject to restriction, the "ideal" result is > that (a) the restrictions are met exactly and (b) whenever a > restriction stipulates a definite numerical value for a > parameter, its variance is exactly zero. But -- in line with > your point above -- in general that ain't gonna happen in > digital arithmetic. In a case like Lee's example, with a bunch > of zero restrictions, we expect to find the computed variances > distributed "randomly" in the close neighbourhood of zero, > with some of them likely negative. In that case I think it's > reasonable to print the standard errors as zero, if they're > close enough, but provide the "true" (numerical, ugly) > variance matrix for those who want to see it. That's now in > CVS. > > Second, when it comes to retrieving the coefficient or > standard error vector from a model, we've checked for NAs and > if any are found we refuse to supply the object (as Lee > observed). OK, that seems too delicate, or paternalistic, or > something. So now in CVS you can access $stderr even if it > contains NAs. > > Allin > I like this solution since, at least for my purposes, it solves an immediate problem. I'm trying to stuff the std errors after a _restrict --full_ statement into a bundle. The function that initiates the bundle fails because $stderr is not returned. I can _catch_ the error and put something into the matrix, but that defeats the purpose of using the _restrict --full_ in this case. I realize the example is extreme, but I'm stress testing the set of functions for a RLS Stein-rule package I'm working on. I probably need to reassess how I'm computing the restricted estimates in order to make the thing backward compatible with previous versions of gretl. My first version used something similar to the Greene example Allin gave, but the restrict function is so elegant I couldn't resist using it instead. Still, being able to put matrices that contain NA into a bundle will pay dividends down the road I think, especially because there are so many accessors available for subsequent use .... gretl does contain several ways to dress these up once they are available (e.g., misszero() ). Thanks, Lee -- Lee Adkins Professor of Economics lee.adkins(a)okstate.edu learneconometrics.com

On Sun, Nov 18, 2012 at 11:42 AM, Lee Adkins <lee.adkins(a)okstate.edu&gt; wrote: > Here is a followup that illustrates what Jack is saying about when is a > small number really zero. Being too strict about the size of the value may > lead to other unintended results. This example is based on the same > dataset (br.gdt) but uses restrictions that are nearly true. > > ? square sqft bedrooms > ? logs price > ? series price = price/100 > ? list xlist = const sqft sq_sqft sq_bedrooms bedrooms baths age > ? matrix Rmat = zeros(3,4)~I(3) > ? matrix r = { 700 ; 400; -10 } > ? ols price xlist > > Model 1: OLS, using observations 1-1080 > Dependent variable: price > > coefficient std. error t-ratio p-value > --------------------------------------------------------------- > const 168.782 216.484 0.7797 0.4358 > sqft -0.758827 0.0741780 -10.23 1.68e-023 *** > sq_sqft 0.000248214 1.03688e-05 23.94 8.12e-102 *** > sq_bedrooms -117.075 19.4308 -6.025 2.32e-09 *** > bedrooms 694.058 138.416 5.014 6.22e-07 *** > baths 379.550 46.2502 8.206 6.48e-016 *** > age -8.34062 1.14878 -7.260 7.40e-013 *** > > Mean dependent var 1548.632 S.D. dependent var 1229.128 > Sum squared resid 4.01e+08 S.E. of regression 611.6813 > R-squared 0.753717 Adjusted R-squared 0.752340 > F(6, 1073) 547.2962 P-value(F) 0.000000 > Log-likelihood -8458.451 Akaike criterion 16930.90 > Schwarz criterion 16965.79 Hannan-Quinn 16944.11 > > ? restrict --full > ? R=Rmat > ? q=r > ? end restrict > > Test statistic: F(3, 1073) = 0.955727, with p-value = 0.412961 > > > Model 2: Restricted OLS, using observations 1-1080 > Dependent variable: price > > coefficient std. error t-ratio p-value > ------------------------------------------------------------------ > const 201.080 88.6191 2.269 0.0235 ** > sqft -0.783296 0.0645078 -12.14 6.88e-032 *** > sq_sqft 0.000250439 9.22345e-06 27.15 4.42e-124 *** > sq_bedrooms -118.625 5.21332 -22.75 6.95e-094 *** > bedrooms 700.000 0.000000 NA NA > baths 400.000 4.64027e-07 8.620e+08 0.0000 *** > age -10.0000 0.000000 NA NA > > > Notice that the std error on sq_sqft squared is very small (but not zero) > and the one on baths (which is technically zero) is only 1 decimal smaller. > If you didn't know that the se is supposed to be zero on a restricted > coefficient (like many of my students) you'd report something that was > obviously wrong. In the original example, the problem was not so much in > the restrictions, but the conditioning of the data themselves, which > remains very bad even in this case of "good" restrictions. It's not clear > to me how sorting this out based on size is possible. Is there a complex > eigenvalue associated with the R*inv(X'X)*R' that might identify which > should be NA? > After thinking about this for a second, there will only be 3 eigenvalues for that matrix, and all of them positive to be sure. Hmmm. The ones for X'X are positive (but some are tiny indicating severe collinearity).

> In this example the computed variance for the baths > coefficient is 2.15e-13, which seems on the big side to be > forced to zero. I think the only way to get this "right" would > be to somehow keep track of which coefficients, if any, are > assigned a definite numerical value by the restriction -- i.e. > look for rows of the R matrix that have only one non-zero > entry?

Yes, I think so, too since the std errors (and coefficients) can always be changed by an arbitrary amount by rescaling y or x or both.

It's almost working. Here is the example: <hansl> open "C:\Program Files (x86)\gretl\data\poe\br.gdt"

oops, so sorry. I thought that since the variances had become positive in the example that the corresponding snapshot was already up. I'm in no hurry, really. Neither of us should be working on Sunday anyway :) Please forgive....

On Sun, Nov 18, 2012 at 2:50 PM, Allin Cottrell <cottrell(a)wfu.edu&gt; wrote: > On Sun, 18 Nov 2012, Lee Adkins wrote: > >> It's almost working. Here is the example: >> >> <hansl> >> >> open "C:\Program Files (x86)\gretl\data\poe\br.gdt" > > Wait a mo... are you building gretl for Windows yoourself from CVS? > If not, you'll have to wait for a new Windows snapshot, which I can > put out tomorrow. Right now, what I described in > http://lists.wfu.edu/pipermail/gretl-devel/2012-November/004138.html > is only in CVS.