Dear gretl users, first of all let me thank you, that you have already provided me with solutions on other topics. Unfortunately, I need your help again. The problem I am now trying to solve is the following. Basically, I want to set up a rolling regression with parameter restrictions (all parameters, except for the constant shall sum to one) and store the coefficient estimates. So far this poses no problem: <hansl> Matrix C={} List indep=indep1..indep10 Smpl 1 36 Loop i=1..360 Ols dep const indep Restrict b[2]+b[3]+b[4]+b[5]+ b[6]+b[7]+b[8]+b[9]+b[10]=1 end restrict Matrix c=coeff' C=C|c Smpl +1 +1 Endloop </hansl> But as you can see, I have lots of regressors and not all might be significant, or depending on the time window the significance will change. I know that I can use the omit --auto function to select only significant coefficient estimates, but here it is, where the problems start: 1.) Assume that during the first window, there are 3 significant coefficients, so that the matrix will have 3 columns. If it should be, that during the next time window, there are, say 4 significant coefficients, then the script breaks down (matrices do not fit). Therefore, I guess I have to reshape the matrix somehow, to allow for the new column. 2.)Assume that during the first window there are 3 significant coefficients (2, 3, 4) and during the next time window there are 3 different significant coefficients (6,7,8). Then, the dimension of the matrix would be correct, but interpreting the matrix of coefficients afterwards in a time series context would not make much sense. To summarize 1.) and 2.), I would need a matrix with 10 columns, where "NA" is entered, if the respective coefficient is insignificant and else the coefficient. So that one can obtain the time series of significant regressors. Date indep1 indep2 ... indep10 1 0,8 0,1 0,1 2 0,6 NA NA 3 0,75 NA 0,05 The third issue arises with the parameter restriction. After omitting insignificant variables, the restriction that coefficients sum to one should still apply. Unfortunately there seems to be no simple shortcut for the restriction (for example restrict sum(coeff(2..n))) , whith n being the last significant coefficient). As I don't know ex-ante which parameters would be significant, I somehow have to dynamically readjust the restriction. Is there a way how one can do it? Thanks in advance for your time answering my questions. Kind regards, Jan

Dear Jack, Thank you for your answer and your useful comments, I see the point your are making. As I am not very well trained in econometrics, I have to apologize for not being more formal or precise in the way I stated my problem. I understand that this is quite bothering. Maybe it is not as simple as I thought or at least not as simple to implement in Gretl. You might have already guessed what I am doing, I am trying to run a style analysis on investment funds in the spirit of William Sharpe (1988, 1992). Therefore it would be ideal to have at every step of the iteration a restricted regression where the insignificant coefficients would be dropped afterwards. Then the model should be re-estimated until there are only significant regressors, whose coefficients sum to one. I have noticed by now, that the omit command works on the unrestricted model only. I thought that using restrict command first, and then the omit command would apply the omit algorithm to the restricted model, because the restriction is some kind of an estimation command. I guess that this would make the iterative process easier. However, restricting the model first and then omitting variables works, but as you state the coefficients don't sum to one anymore, which means that the model should be re-estimated so that the restriction holds again. The next best solution, which corresponds to your third alternative (estimate U, drop, restrict), would be to estimate the unrestricted model first, then omit the insignificant coefficients, then re-estimate the reduced model with the portfolio constraint. It might be that once significant coefficient become insignificant after restricting the portfolio weights, but I would guess that it is unlikely, because the sensitivity of a fund's return to a factor would remain, even if I restrict the regression, wouldn't it? Nonetheless, if there should be any insignificant coefficients after restricting the reduced model, then the process shall start again. One problem that remains is, that you don't know exante which coefficients will become insignificant during the iteration. Therefore the restriction has to be "dynamic", which I tried to implement via an if..elif..else..endif block within the restriction block. Unfortunately, this did not work (parsing error). Furthermore, I want as an output the time series of significant coefficients only. Therefore I need a matrix (T by n), where the n-coefficients are always written into the same column, depending on the name of the regressor and not on the coefficient number, as the coefficient b[2] can vary throughout time due to restricting the model. I know that if I would do Sharpe's style "econometrically correct", I would select the indexes that span the manager's investment universe up front and do a check for (near) multicollinearity, so that the results would be sensible. In that case I could end here and start the constrained rolling regression, which would yield a time series of average "portfolio-weights". Again, I really appreciate your comments and I am aware that I am mining the data so that the results might be spurious. Kind regards, Jan -----Ursprüngliche Nachricht----- Von: Plus.line MailSystem [mailto:cyrus@mailer.plusline.de] Im Auftrag von Riccardo (Jack) Lucchetti Gesendet: Freitag, 31. August 2012 15:40 An: Gretl list Betreff: Re: [Gretl-users] data mining with rolling regression and restricted coefficients On Wed, 29 Aug 2012, Jan Tille wrote: No problem, that's what the mailing list is for. [... rolling regression with constraints ...] IMHO, your problem can be solved, once it's stated less ambiguously. Allow me to explain (and to apologise in advance for being pedantic). Suppose you have 3 regressors instead of 10. Then you have y_t = b0 + x_{1t} b1 + x_{2t} b2 + x_{3t} b3 + u_t Let's call this model U (for unrestricted). Once you impose the restriction b1+b2+b3=1, you can write the same model as y_t - x_3 = b0 + (x_{1t} - x_{3t}) b1 + (x_{2t} - x_{3t}) b2 + u_t Call this model R (restricted). Running OLS on model R, you get exactly the constrained estimates you're after. Then, you can recover you estimate of b3 as 1 - b1 - b2 and compute its standard error as sqrt(V(b1+b2)), which is easy to do. Do you want to NOW drop those coefficients whose t-statistic is less than a predefined threshold? But in this case, the remaining ones won't sum to 1 anymore. Or perhaps, you want run model R, trim it to your liking and then compute b3 as 1 - (sum of surviving betas). But then, what guarantee do you have that that b3 will itself be significant? Because if it isn't, you have the option of dropping it (and have the sum being different from 1) or keeping it (but why should you treat b3 specially?). Or maybe you want to run model U first, drop the "excess" variables and then force the coefficients to sum to 1? Because in this case, you have no guarantee that the surviving coefficients will be "significant" (whatever that means). I don't mean to be patronizing, but I think that before you think of the hansl syntax to do what you want, you should think a little harder about what you actually want! :) -------------------------------------------------- Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche (formerly known as Università di Ancona) r.lucchetti(a)univpm.it http://www2.econ.univpm.it/servizi/hpp/lucchetti --------------------------------------------------

Allin, Thank you - I hope that I got it right now, when I say that it is preferable . As wrote in the mail to Jack, then it is preferable to omit insignificant variables in the first step and re-estimate the reduced model with the portfolio constraint. That should work quite well. Nonetheless I am struck with the problem, that I don't know ex-ante how many insignificant variables might be dropped. I guess, that is what you mean by "you need to decide how you're going to respecify the restriction in case some terms are not statistically significant". If I assume the 3-Variable case, then in the worst case three variables (four including the constant) might be dropped. To control for this and to respecify the restriction after omitting the variables, I implemented the if-block below: matrix CO={} # ---- estimate unrestricted model----- ols y const x1 x2 x3 scalar kfull=$ncoeff #---- omit insignificant variables---- omit --auto #---determine how many coefficients are left---- scalar k=kfull-$ncoeff restrict if k=0 #---if all variables are significant b[2]+b[3]+b[4]= 1 elif k=1 #----if one variable has been dropped b[2]+b[3] 1 else k=2 #----if two variables have been dropped b[2]=1 endif end restrict matrix co = $coeff' CO = CO | co Still, if I want to put it into a rolling regression, then I would need to set up the coefficient matrix correctly. Kind regards, Jan -----Ursprüngliche Nachricht----- Von: Plus.line MailSystem [mailto:cyrus@mailer.plusline.de] Im Auftrag von Allin Cottrell Gesendet: Dienstag, 4. September 2012 14:09 An: Gretl list Betreff: Re: [Gretl-users] data mining with rolling regression and restricted coefficients On Tue, 4 Sep 2012, Jan Tille wrote: I think Jack's point was the other way round: the procedure you're talking could be interpreted in various ways so it's hard to know what count as "right". But having decided on that, it shouldn't be difficult to implement in gretl. Your procedure illustrates why this is not possible in general. If the "omit" command drops a variable that participated in the prior restriction (e.g. that the coefficients sum to 1), then what is gretl supposed to do? Rather than simply using "omit", you need to decide how you're going to respecify the restriction in case some terms are not statistically significant. Allin Cottrell _______________________________________________ Gretl-users mailing list Gretl-users(a)lists.wfu.edu http://lists.wfu.edu/mailman/listinfo/gretl-users

On Tue, 4 Sep 2012, Allin Cottrell wrote: Exactly. Sorry if I wasn't clear. -------------------------------------------------- Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche (formerly known as Università di Ancona) r.lucchetti(a)univpm.it http://www2.econ.univpm.it/servizi/hpp/lucchetti --------------------------------------------------

Dear Charles, Thank you for your comments. As you state, it is not a-priori known which coefficients would be significant. I wrote to Jack, that I tried to use an if-block to allow for this problem, which unfortunately did not work. Anyway I would still be left with the problem of how to create a matrix of dimension (T by n), where the coefficients of a regressor are always in the same column. If I use the structure below, then the script brakes down if I start with, say 2 significant regressors, and within the next time window there are three significant regressors. matrix CO3={} . . . matrix co = $coeff' CO3 = CO3 | co Kind regards, Jan Von: Plus.line MailSystem [mailto:cyrus@mailer.plusline.de] Im Auftrag von Charles Ward Gesendet: Sonntag, 2. September 2012 17:24 An: Gretl list Betreff: Re: [Gretl-users] data mining with rolling regression and restricted coefficients One trick to use for this type of problem is to transform the regression. So if we want to regress y against x1, x2 and x3 and constrain the parameters to sum to 1 we do the following, Subtract x1 from each of the other variables so we have y-x1, x2-x1, x3-x1 Regress y-x1 against the other two variables (no constraints needed unless you want all the parameters to be positive). y-x1 = A + B(x2-x1) + C(x3-x1) +e Rearranging the results gives us y = A + x1(1-B-C) + Bx2 + Cx3 +e Thus the parameters sum to 1. The only problem in your case is that it would be easier to program if you have one variable (x1 in the above case) that is always present on the RHS of the regression. Charles Ward http://www.icmacentre.ac.uk/person/professor-charles-ward On 29 August 2012 14:38, Jan Tille <Jan.Tille@absolut-research.de<mailto:Jan.Tille@absolut-research.de>> wrote: Dear gretl users, first of all let me thank you, that you have already provided me with solutions on other topics. Unfortunately, I need your help again. The problem I am now trying to solve is the following. Basically, I want to set up a rolling regression with parameter restrictions (all parameters, except for the constant shall sum to one) and store the coefficient estimates. So far this poses no problem: <hansl> Matrix C={} List indep=indep1..indep10 Smpl 1 36 Loop i=1..360 Ols dep const indep Restrict b[2]+b[3]+b[4]+b[5]+ b[6]+b[7]+b[8]+b[9]+b[10]=1 end restrict Matrix c=coeff' C=C|c Smpl +1 +1 Endloop </hansl> But as you can see, I have lots of regressors and not all might be significant, or depending on the time window the significance will change. I know that I can use the omit --auto function to select only significant coefficient estimates, but here it is, where the problems start: 1.) Assume that during the first window, there are 3 significant coefficients, so that the matrix will have 3 columns. If it should be, that during the next time window, there are, say 4 significant coefficients, then the script breaks down (matrices do not fit). Therefore, I guess I have to reshape the matrix somehow, to allow for the new column. 2.)Assume that during the first window there are 3 significant coefficients (2, 3, 4) and during the next time window there are 3 different significant coefficients (6,7,8). Then, the dimension of the matrix would be correct, but interpreting the matrix of coefficients afterwards in a time series context would not make much sense. To summarize 1.) and 2.), I would need a matrix with 10 columns, where "NA" is entered, if the respective coefficient is insignificant and else the coefficient. So that one can obtain the time series of significant regressors. Date indep1 indep2 ... indep10 1 0,8 0,1 0,1 2 0,6 NA NA 3 0,75 NA 0,05 The third issue arises with the parameter restriction. After omitting insignificant variables, the restriction that coefficients sum to one should still apply. Unfortunately there seems to be no simple shortcut for the restriction (for example restrict sum(coeff(2..n))) , whith n being the last significant coefficient). As I don't know ex-ante which parameters would be significant, I somehow have to dynamically readjust the restriction. Is there a way how one can do it? Thanks in advance for your time answering my questions. Kind regards, Jan _______________________________________________ Gretl-users mailing list Gretl-users@lists.wfu.edu<mailto:Gretl-users@lists.wfu.edu> http://lists.wfu.edu/mailman/listinfo/gretl-users