Am 18.07.2011 22:00, schrieb Allin Cottrell: Sounds reasonable, although in that area there may be subtleties, so I'm not 100% sure I understand the implications. Also, in combination with the "reachback" to pre-sample values that we discussed earlier, couldn't there be some undesired side effect? Say the subsample starts at obs 5, and you have two lags, so gretl would normally include obs 3 and 4. But imagine obs 3 is missing. If the function author asks firstobs(), currently she would get obs 4 as the answer (right?). In the future, the answer would be obs 5, if I understand correctly. The author would not be able to handle the missings in the pre-sample values. (Or at least it would be more difficult.) I'm not saying the change of firstobs() is bad, rather to me this looks like a further indication that the pre-sample reachback behavior may be problematic. cheers, sven

On Mon, 18 Jul 2011, Allin Cottrell wrote: The more I think about this, the more I get confused. Let me start from the easy part: Allin's proposal makes a lot of sense to me, and I am 100% in favour. If you absolutely need to know what data you have regardless of the current sample range, do a "smpl full" before the real work begins, store the result into two scalars and go ahead. This said, I've begun to think about the usefulness of having those two functions at all. Clearly, they're no use for cross-section data. How about panel datasets? Either they return something entirely different (vectors, maybe?) or they're no use in this case, either. Moreover: I would guess that what most people would use firstobs() and lastobs() for is some sort of loop-based algorithm which deals with time-series data. In that case, either you're absolutely sure you never get NAs between firstobs() and lastobs() or you need to put some sort of check in place. But if you do, then what's the use of firstobs() and lastobs()? Note: I'm not making a definite proposal (other than agreeing with Allin's original point). I'd just like to hear your opinions on this. Riccardo (Jack) Lucchetti Dipartimento di Economia Università Politecnica delle Marche r.lucchetti(a)univpm.it http://www.econ.univpm.it/lucchetti

On Tue, 19 Jul 2011, Riccardo (Jack) Lucchetti wrote: No quite true, see below. First of all, these functions are rather old and were arguably not very well thought out in the first place. They give you the first or last valid observation in the dataset, but they don't guarantee that the range thus defined contains no NAs. However, they can serve as the basis for a check, as in <hansl> t1 = firstobs(y) t2 = lastobs(y) smpl t1 t2 if sum(missing(y) > 0 # error condition endif </hansl> Note that a check such as this is probably most common for time series data, but it's also relevant if you want to apply an estimator that can't handle "internal" NAs, on any sort of data. But we now have better ways of doing this check, for example: <hansl> catch smpl y --contiguous if $error funcerr "found internal NAs" endif </hansl> At this point I'm inclined in Berend's direction: maybe if we're going to keep firstobs() and lastobs() we should leave their sematics unchanged, because that way they do something that smpl --contiguous can't do, and that might be useful in some cases. Allin

1) I agree in that firstobs() and lastobs() may restrict their checks to the current sample range. 2) With respect to comments from Jack and Sven, I agree with Sven in that normally if you work with time series you will have contiguous data. There is no need to check for missing data inside your current sample, because what if you have any? gretl will manage this observation as an NA and you will see the effect in your results, or maybe you will obtain an error in your subsequent calculations and you will have the oportunity to have that into account. 3) But I think we should have a lot of care with backward incompatible changes. I found in some cases that my old (working) scripts does not work now, and in such a case you have to loose a lot of time making them to work. 4) I have been using lately cross section data (which by the way is very rare on me), and I admit the relevant command in this case is "smpl --no-missing" in my case, using the option "smpl --no-missing list_of variables" was very useful. -- Ignacio Diaz-Emparanza DEPARTAMENTO DE ECONOMÍA APLICADA III (ECONOMETRÍA Y ESTADÍSTICA) UPV/EHU Avda. Lehendakari Aguirre, 83 | 48015 BILBAO T.: +34 946013732 | F.: +34 946013754 www.ea3.ehu.es