12:43 p.m.
Just out of curiosity: Does it make any difference, from a computational
perspective, whether I choose option 1 or 2? In option 2 I concretely
pre-allocate memory, which should be more efficient, right?
<hansl>
#Opt 1:
matrix X = x1~x2~..~xp
#Opt 2:
matrix X = zeros(T,p)
X[,1] = x1
X[,2] = x2
...
X[,p] = xp
<\hansl>
Artur
12:53 p.m.
Am 27.03.2014 13:43, schrieb Artur T.:
Just out of curiosity: Does it make any difference, from a
computational
perspective, whether I choose option 1 or 2? In option 2 I concretely
pre-allocate memory, which should be more efficient, right?
<hansl>
#Opt 1:
matrix X = x1~x2~..~xp
#Opt 2:
matrix X = zeros(T,p)
X[,1] = x1
X[,2] = x2
...
X[,p] = xp
<\hansl>
Only a test will tell, but my intuition would be that the overhead of
writing all those zeroes into the initial matrix and then also
interpreting the indexing will be much worse than just concatenating.
I also would be surprised if the memory allocation mattered for such an
interpreted language like hansl, but I guess this ultimately depends on
the implementation under the hood.
-sven
1:51 p.m.
I ran a small script, and indeed option 1 is slightly faster.
Artur
2014-03-27 13:53 GMT+01:00 Sven Schreiber <svetosch(a)gmx.net>:
Am 27.03.2014 13:43, schrieb Artur T.:
> Just out of curiosity: Does it make any difference, from a computational
> perspective, whether I choose option 1 or 2? In option 2 I concretely
> pre-allocate memory, which should be more efficient, right?
>
> <hansl>
> #Opt 1:
> matrix X = x1~x2~..~xp
>
> #Opt 2:
> matrix X = zeros(T,p)
> X[,1] = x1
> X[,2] = x2
> ...
> X[,p] = xp
> <\hansl>
>
Only a test will tell, but my intuition would be that the overhead of
writing all those zeroes into the initial matrix and then also
interpreting the indexing will be much worse than just concatenating.
I also would be surprised if the memory allocation mattered for such an
interpreted language like hansl, but I guess this ultimately depends on
the implementation under the hood.
-sven
2:16 p.m.
On Thu, Mar 27, 2014 at 12:43 PM, Artur T.
<artur.tarassow(a)googlemail.com> wrote:
Just out of curiosity: Does it make any difference, from a
computational
perspective, whether I choose option 1 or 2? In option 2 I concretely
pre-allocate memory, which should be more efficient, right?
<hansl>
#Opt 1:
matrix X = x1~x2~..~xp
#Opt 2:
matrix X = zeros(T,p)
X[,1] = x1
X[,2] = x2
...
X[,p] = xp
<\hansl>
I would expect option 2 to be more efficient for large matrices: it
should be less
time consuming to fill a pre-allocated matrix that to keep resizing a
matrix that
starts out dimensionless. But Sven is right (in his parallel reply) to
say that it's
an empirical matter.
Having done an experiment I'm actually quite surprised at just how much faster
option 2 is. Here's my test:
<hansl>
scalar p = 400
matrix xi = mnormal(T,1)
# option 1
set stopwatch
matrix X = {}
loop i=1..p -q
X ~= xi
endloop
printf "time1 = %g seconds\n", $stopwatch
# option 2
set stopwatch
matrix X = zeros(T,p)
loop i=1..p -q
X[,i] = xi
endloop
printf "time2 = %g seconds\n", $stopwatch
</hansl>
I've run this several times (and also with the order swapped) and I'm seeing
about 0.4 seconds for option 1 versus 0.007 seconds for option 2.
Allin Cottrell
2:44 p.m.
On Thu, 27 Mar 2014, Cottrell, Allin wrote:
Having done an experiment I'm actually quite surprised at just
how much faster
option 2 is. Here's my test:
You shuld also consider the "loop" overhead. Try this:
<hansl>
set echo off
set messages off
loop i=1..10 --quiet
matrix X$i = mnormal(200,20)
endloop
# option 1
set stopwatch
loop 100 --quiet
matrix A = X1 ~ X2 ~ X3 ~ X4 ~ X5 ~ X6 ~ X7 ~ X8 ~ X9 ~ X10
endloop
t0 = $stopwatch
set stopwatch
loop 100 --quiet
matrix A = {}
loop i=1..10 --quiet
A = A ~ X$i
endloop
endloop
t1 = $stopwatch
print t0 t1
</hansl>
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
11:45 a.m.
Thank you for the replies. As you said, the significance really depends on
the size of the matrices. But the efficient matrix pre-allocation seems to
be a cleaner way of programming as well -- at least to me :-)
Best,
Artur
2014-03-27 15:44 GMT+01:00 Riccardo (Jack) Lucchetti <r.lucchetti(a)univpm.it>
:
On Thu, 27 Mar 2014, Cottrell, Allin wrote:
Having done an experiment I'm actually quite surprised at just how much
> faster
> option 2 is. Here's my test:
>
You shuld also consider the "loop" overhead. Try this:
<hansl>
set echo off
set messages off
loop i=1..10 --quiet
matrix X$i = mnormal(200,20)
endloop
# option 1
set stopwatch
loop 100 --quiet
matrix A = X1 ~ X2 ~ X3 ~ X4 ~ X5 ~ X6 ~ X7 ~ X8 ~ X9 ~ X10
endloop
t0 = $stopwatch
set stopwatch
loop 100 --quiet
matrix A = {}
loop i=1..10 --quiet
A = A ~ X$i
endloop
endloop
t1 = $stopwatch
print t0 t1
</hansl>
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
_______________________________________________
Gretl-users mailing list
Gretl-users(a)lists.wfu.edu
http://lists.wfu.edu/mailman/listinfo/gretl-users
3674
days inactive
3676
days old
5 comments
4 participants
participants (4)
-
Artur T. -
Cottrell, Allin -
Riccardo (Jack) Lucchetti -
Sven Schreiber