[Gretl-users] Re: One large matrix or many small matrices for storing simulation results?
On Fri, 23 Apr 2021, Alecos Papadopoulos wrote:
Thanks for both good advises, fortunately I have been following them
all
along, namely, declare the matrices before hand, and if slice, then slice by
column.
However, none replies to my question. So I run an experiment based on Jack's
example
<hansl>
#case 1
X = zeros(N, 3)
set stopwatch
loop i = 1..N
X[i,1] = sqrt(i)
X[i,2] = sqrt(i)
X[i,3] = sqrt(i)
endloop
t1 = $stopwatch
#case 2
X1 = zeros(N, 1)
X2 = zeros(N, 1)
X3 = zeros(N, 1)
set stopwatch
loop i = 1..N
X1[i,1] = sqrt(i)
X2[i,1] = sqrt(i)
X3[i,1] = sqrt(i)
endloop
t2 = $stopwatch
</hansl>
and I got
<output>
t1 = 0.641354
t2 = 0.653886
</output>
So, perhaps contrary to first a priori impressions, it appears that slicing
down the column dimension, if anything, makes gretl slow down a bit.
When timings are as close as in this case, you should try executing
the two variants in both orders. With very intensive computation the
registers will heat up and run faster by the time you get to the
second variant.
Cutting your big matrix into a set of column vectors can simplify
the indexation, but you're (partially) missing out on that by
providing a redundant second index: skip that, and you may find the
column-vectors case runs slightly faster.
I replicated your example with N = 100000; gave just the row index
in the column-vectors case, as in
X1[i] = sqrt(i)
and named the times t_mat (using a single matrix) and t_vec (using
per-column vectors). What I'm seeing is:
t_vec 0.423s
t_mat 0.441s
t_vec 0.420s
There's not much in it, but the set-of-vectors approach seems
marginally faster.
I very much doubt you'd see a speed-up if your "slicing" didn't
yield vectors (that is, if it just reduced an m x n matrix to a set
of m x k matrices with k > 1), since then there would be no
reduction in indexation overhead.
However, it we take the computational task to be literally what's in
your example (which we probably don't want to do!), there's a MUCH
faster way of doing it in whole-matrix mode: precompute a vectorized
solution:
matrix y = sqrt(seq(1,N))'
loop j = 1..3
X[,j] = y
endloop
The timing for that with N = 1000000 is 0.0149 seconds.
Allin