11:21 a.m.
Hi all,
I'm trying to implement a rolling regression using the following
specification:
loop r = 1 .. rolls
loop foreach i share
if sum(missing(share.$i)) == 0
lad share.$i const index
series beta.$i[$t2] = $coeff[2]
endif
endloop
if r<rolls
smpl +step +step
endif
endloop
It basically works well with OLS, but not with LAD. Strangely, the
script output hangs and doesn't end and I have to force quit the
program. Any idea why the specification doesn't work with LAD?
Thanks in advance,
--
Filipe Rodrigues da Costa
Send me an email to: filipe(a)pobox.io
Reach me through Telegram at: https://t.me/rodriguesdacosta
11:36 a.m.
Am 11.12.2017 um 17:21 schrieb Filipe Rodrigues da Costa:
Hi all,
I'm trying to implement a rolling regression using the following
specification:
loop r = 1 .. rolls
loop foreach i share
if sum(missing(share.$i)) == 0
lad share.$i const index
series beta.$i[$t2] = $coeff[2]
endif
endloop
if r<rolls
smpl +step +step
endif
endloop
Hi Filipe, would be good if you could send a complete example script
using some dataset which is available to all.
Thanks,
Artur
12:58 p.m.
Ok, here it goes:
-------------------------------------------
open data9-13.gdt
list returns = *ret
# set some parameters for the rolling regression
scalar step = 1 # regression step
scalar size = 12 # regression size
scalar rolls = floor(($nobs - size + 1) / step) #number of rolls
scalar firstobs = $t1
# generate empty beta series
loop foreach i returns
series beta$i = NA
endloop
# set the initial sample
smpl firstobs firstobs+size-1
loop r = 1 .. rolls
loop foreach i returns
if sum(missing(returns.$i)) == 0
lad returns.$i const sp500
series beta$i[$t2] = $coeff[2]
endif
endloop
if r<rolls
smpl +step +step
endif
endloop
smpl --full
---------------------------------------------------------
In the example both OLS and LAD seem to work well. The problem is that
with my large data files, LAD doesn't work.
--
Filipe Rodrigues da Costa
Send me an email to: filipe(a)pobox.io
Reach me through Telegram at: https://t.me/rodriguesdacosta
On Mon, 11 Dec 2017, at 16:36, Artur Tarassow wrote:
Am 11.12.2017 um 17:21 schrieb Filipe Rodrigues da Costa:
> Hi all,
>
> I'm trying to implement a rolling regression using the following
> specification:
>
> loop r = 1 .. rolls
> loop foreach i share
> if sum(missing(share.$i)) == 0
> lad share.$i const index
> series beta.$i[$t2] = $coeff[2]
> endif
> endloop
> if r<rolls
> smpl +step +step
> endif
> endloop
>
Hi Filipe, would be good if you could send a complete example script
using some dataset which is available to all.
Thanks,
Artur
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
1:25 p.m.
On Mon, 11 Dec 2017, Filipe Rodrigues da Costa wrote:
Ok, here it goes:
-------------------------------------------
open data9-13.gdt
list returns = *ret
# set some parameters for the rolling regression
scalar step = 1 # regression step
scalar size = 12 # regression size
scalar rolls = floor(($nobs - size + 1) / step) #number of rolls
scalar firstobs = $t1
# generate empty beta series
loop foreach i returns
series beta$i = NA
endloop
# set the initial sample
smpl firstobs firstobs+size-1
loop r = 1 .. rolls
loop foreach i returns
if sum(missing(returns.$i)) == 0
lad returns.$i const sp500
series beta$i[$t2] = $coeff[2]
endif
endloop
if r<rolls
smpl +step +step
endif
endloop
smpl --full
---------------------------------------------------------
In the example both OLS and LAD seem to work well. The problem is that
with my large data files, LAD doesn't work.
I tried modifying your example above, replacing
open data9-13.gdt
list returns = *ret
with
nulldata 1000
list returns
loop i=1..50
returns += genseries(sprintf("x%dret", i), normal())
endloop
series sp500 = normal()
but leaving the rest unchanged. So that's 1000 observations on 50
returns and the script still ran OK. Roughly what are the actual
dimensions of your data?
Additional thought: you might try replacing
lad y const x
with
quantreg 0.5 y const x
That should probably be faster for large data.
Allin
1:33 p.m.
On Mon, 11 Dec 2017, Allin Cottrell wrote:
I tried modifying your example above, replacing
open data9-13.gdt
list returns = *ret
with
nulldata 1000
list returns
loop i=1..50
returns += genseries(sprintf("x%dret", i), normal())
endloop
series sp500 = normal()
but leaving the rest unchanged. So that's 1000 observations on 50
returns and the script still ran OK. Roughly what are the actual
dimensions of your data?
Ah, silly me. Cranking up the overall sample length and the number of
returns will each just scale the time taken linearly. What might
really matter for lad is the number of observations used in each
regression (the "size" variable in your script).
I suspect if your real "size" value is quite large lad may be bogging
down, and in that case quantreg may do better.
Allin
2:24 p.m.
Hi Allin,
I have 27 years of daily price data. I was running rolling regressions
of 252 data points. For a small country I got 2 million OLS, so, I do
have a large amount of data.
When I ran OLS rolling regression, I notice the computer RAM in use
increasing fast. After some time, the 8GB of physical memory is
completely used by Gretl and some swap memory is additionally used. But,
when I try the same for LAD, memory use stops increasing, giving the
idea that and error occurs at the earliest stages.
I will try your sugestions in my data and let you know if it was
successful.
--
Filipe Rodrigues da Costa
Send me an email to: filipe(a)pobox.io
Reach me through Telegram at: https://t.me/rodriguesdacosta
On Mon, 11 Dec 2017, at 18:33, Allin Cottrell wrote:
On Mon, 11 Dec 2017, Allin Cottrell wrote:
> I tried modifying your example above, replacing
>
> open data9-13.gdt
> list returns = *ret
>
> with
>
> nulldata 1000
> list returns
> loop i=1..50
> returns += genseries(sprintf("x%dret", i), normal())
> endloop
> series sp500 = normal()
>
> but leaving the rest unchanged. So that's 1000 observations on 50
> returns and the script still ran OK. Roughly what are the actual
> dimensions of your data?
Ah, silly me. Cranking up the overall sample length and the number of
returns will each just scale the time taken linearly. What might
really matter for lad is the number of observations used in each
regression (the "size" variable in your script).
I suspect if your real "size" value is quite large lad may be bogging
down, and in that case quantreg may do better.
Allin
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
5:39 a.m.
I must admit that replacing
lad y const x
for
quantreg 0.5 y const x
works like a charm!
The script is slower than when using OLS but manageable for an execution
of 2 million regressions!
Thanks
--
Filipe Rodrigues da Costa
Send me an email to: filipe(a)pobox.io
Reach me through Telegram at: https://t.me/rodriguesdacosta
Additional thought: you might try replacing lad y const x with
quantreg 0.5 y const x That should probably be faster for large
data. Allin
7:28 a.m.
Am 13.12.2017 um 11:39 schrieb Filipe Rodrigues da Costa:
I must admit that replacing
> lad y const x
for
> quantreg 0.5 y const x
works like a charm!
Interesting -- this kind of begs the question (not to you, Filipe) why
'lad' isn't automatically using quantreg internally, perhaps after some
heuristic threshold number of observations?
thanks,
sven
1:42 p.m.
On Wed, 13 Dec 2017, Sven Schreiber wrote:
Am 13.12.2017 um 11:39 schrieb Filipe Rodrigues da Costa:
> I must admit that replacing
>
>> lad y const x
> for
>> quantreg 0.5 y const x
>
> works like a charm!
Interesting -- this kind of begs the question (not to you, Filipe) why 'lad'
isn't automatically using quantreg internally, perhaps after some heuristic
threshold number of observations?
Actually "lad" does use quantreg internally. But the quantreg code
comprises two algorithms, Barrodale-Roberts and Frisch-Newton, and the
lad command selects Barrodale-Roberts, which is slower for big data.
And in addition, lad calls for bootstrapped standard errors. The
reason for these choices is historical: it matches how "lad" used to
behave before quantreg came along.
I strongly suspect that what really slowed things down for Felipe was
the bootstrapping of standard errors.
Anyway, I guess it's time to switch lad to Frisch-Newton and not
bootstrap by default, though we could add options to do those things.
Allin
2484
days inactive
2488
days old
8 comments
4 participants
participants (4)
-
Allin Cottrell -
Artur Tarassow -
Filipe Rodrigues da Costa -
Sven Schreiber