10:36 a.m.
Hi folks, greetings from Paris !
I am currently working on a spatial statistics project. I intend to develop
progressively some gretl functions to compute spatial correlation tests.
A difficulty is to handle spatial neighborhood matrices that are needed to
compute those tests. More specifically, I have an external file storing the
matrix in vector form: in the first column the list of the spatial units, and
in the second the list of their neighbors. The difficulty is that having two
different datafiles of different dimensions does not easily allow to pass the
contiguity matrix into the datafile containing the data to be analysed.
I guess it could be done with a null list and an append command into a loop,
provided a recoding is done so that the neighbor matrix is recoded eg, in the
following format:
Obs No Neighbor1 Neighbor2 Neighbor3 ... Neighbor n
1 nn1 nn2 nn3 nnN
2 nn1
...
n
where nn1-nnN is the Observation No of unit n
In this case handling is perfect. For now, I use a third party application to
do that. But would like to avoid that but I do not succeed in opening the
contiguity matrix file, recode it and store inot the datafile.
Any advice ?
Two, do the matrix commands support the boolean product command ? It would be
very useful for contiguity matrix handling purposes (ie, to compute the
neighbors of neighbors, etc...).
Three, if you are interested in this topic, I will gladly contribute my
developments to the community.
Cheers !
Franck
--
Franck Nadaud
Economiste
CIRED
UMR 8568 CNRS - EHESS
45 bis avenue de la Belle Gabrielle
94736 Nogent-sur-Marne Cedex
TEL 33-1-43-94-73-94
FAX: 33-1-43-94-73-70
MOB: 06-07-39-92-75
France
9:54 p.m.
On Wed, 28 Nov 2007, Franck Nadaud wrote:
Hi folks, greetings from Paris !
I am currently working on a spatial statistics project. I intend to develop
progressively some gretl functions to compute spatial correlation tests.
This would be brilliant.
A difficulty is to handle spatial neighborhood matrices that are
needed to
compute those tests. More specifically, I have an external file storing the
matrix in vector form: in the first column the list of the spatial units, and
in the second the list of their neighbors. The difficulty is that having two
different datafiles of different dimensions does not easily allow to pass the
contiguity matrix into the datafile containing the data to be analysed.
One of the thing planned for future versions is a function for
reading/writing matrices from/to csv files. Would this be useful to you?
I guess it could be done with a null list and an append command into
a loop,
provided a recoding is done so that the neighbor matrix is recoded eg, in the
following format:
Obs No Neighbor1 Neighbor2 Neighbor3 ... Neighbor n
1 nn1 nn2 nn3 nnN
2 nn1
...
n
where nn1-nnN is the Observation No of unit n
In this case handling is perfect. For now, I use a third party application to
do that. But would like to avoid that but I do not succeed in opening the
contiguity matrix file, recode it and store inot the datafile.
Any advice ?
I'm not sure I understand exactly what you mean, but in the (unlikely)
case I do, you may use the mshape() function to put thing into place and
then convert the columns of the resulting matrix to series.
Two, do the matrix commands support the boolean product command ? It would be
very useful for contiguity matrix handling purposes (ie, to compute the
neighbors of neighbors, etc...).
What's the boolean product? You mean something like this?
<script>
nulldata 3
set seed 4385
matrix a = muniform(6,1)
matrix b = muniform(1,3)
matrix A = a > 0.5
matrix B = b > 0.5
A
B
matrix C = A*B
C
</script>
Three, if you are interested in this topic, I will gladly contribute
my
developments to the community.
Please do!
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
10:02 a.m.
Hello listers ! greetings from Paris !
This would be brilliant.
Sure it would be useful in some contexts. I am on the way to convert some old
scripts I wrote in SPSS matrix processing language some (long) time ago. I
plan to propose the Moran and Geary spatial correlation coefficients first.
They are used as residuals mispecification tests when a regression model uses
data related to a spatial context (States, counties, measure stations,
etc...).
One of the thing planned for future versions is a function for
reading/writing matrices from/to csv files. Would this be useful to you?
That would be exactly what is needed for the task. In fact most spatial
analysis toolboxes / softwares proceed exactly that way around.
The data and the spatial context are stored in different files. For example,
in SPSS I had a SAV file and a text file with the neighborhood in vectorized
form.
First I load the data and the neighborhood vector, then I expand it into a
contiguity matrix (a binary matrix C, where C[i,j] = 1 if spatial unit j is a
neighbor of i, zero if not). Then I use the built contiguity matrix to compute
spatial statistics and perform tests.
The proposition you mentioned would do perfectly the job. The problem I had is
that the formats of the data and neighbor matrix are different so it is not
possible to load the data without loosing the matrix because only one datafile
can be opened at the same time!
So i had to translate my contiguity vector into a format that would allow it
to be inserted in the datafile. Let me more specific :
Let we have N spatial units (counties), indexed by i = 1..N
The spatial context may be stored in very diverse manners (I will write a
short note on that). The most compact form is the vectorized list of
neighbors: for each of the N spatial units we list only the index of the
neighbors (non zero entries). We thus have
INDEX | NEIGHBORS INDEX
1 9
1 102
1 45
1 33
...
N 12
N 99
N 118
N 78
Here i consider the 4-nearest neighbor matrix (the 4 nearest neighbors of each).
County 1 has county 9, 102, 45 and 33 as neighbors, etc...
this matrix is (4*N;2) so i had to transform it by third party application to
insert it into the data file:
INDEX NEIB1 NEIB2 NEIB3 NEIB4
1 9 102 45 33
2 ...
...
N 12 99 118 78
Then i can perform tests in GRETL.
I'm not sure I understand exactly what you mean, but in the
(unlikely)
case I do, you may use the mshape() function to put thing into place and
then convert the columns of the resulting matrix to series.
Oh right ! Sure I try it, may be faster than loops.
I will develop my sripts in very small example whose results are certified,
from papers and books.
>
> Two, do the matrix commands support the boolean product command ? It would
> be
> very useful for contiguity matrix handling purposes (ie, to compute the
> neighbors of neighbors, etc...).
What's the boolean product? You mean something like this?
<script>
nulldata 3
set seed 4385
matrix a = muniform(6,1)
matrix b = muniform(1,3)
matrix A = a > 0.5
matrix B = b > 0.5
A
B
matrix C = A*B
C
</script>
It seems so, it is simply AND, OR and things like that applied to matrices.
You know it is useful to compute unbiased higher order contiguity matrices
(ie; neighbors of neighbors, etc...).
> Three, if you are interested in this topic, I will gladly
contribute my
> developments to the community.
Please do!
Let's go on folks ! What about a small script with a short paper describing
what is done and why with references ?
cheers
Franck
2:31 p.m.
El Thursday 29 November 2007 11:02:03 Franck Nadaud escribió:
Hello listers ! greetings from Paris !
> This would be brilliant.
Sure it would be useful in some contexts. I am on the way to convert some
old scripts I wrote in SPSS matrix processing language some (long) time
ago. I plan to propose the Moran and Geary spatial correlation coefficients
first. They are used as residuals mispecification tests when a regression
model uses data related to a spatial context (States, counties, measure
stations, etc...).
> One of the thing planned for future versions is a function for
> reading/writing matrices from/to csv files. Would this be useful to you?
That would be exactly what is needed for the task. In fact most spatial
analysis toolboxes / softwares proceed exactly that way around.
There is'nt actually a standard procedure for saving and reading matrices,
but Allin was working about this subject some months ago. You can read his
message to the gret-devel list in
http://lists.wfu.edu/pipermail/gretl-devel/2007-April/000326.html
Probably I should not recommend this tricky procedure to everyone, but ....
I do it in this way:
To save the matrix:
<script>
set echo off
outfile --write "myfile.xml"
print "<?xml version="1.0"?>"
print "<gretl-matrices count="1">"
print "<gretl-matrix name="mymatrix" rows="3"
cols="2">"
print "1.3 2.0"
print "3.4 2.7"
print "8.2 3.1"
print "</gretl-matrix>"
print "</gretl-matrices>"
outfile --close
set echo on
</script>
To read the matrix:
include myfile.xml
--
Ignacio Diaz-Emparanza
DEPARTAMENTO DE ECONOMÍA APLICADA III (ECONOMETRÍA Y ESTADÍSTICA)
UPV/EHU
Avda. Lehendakari Aguirre, 83 | 48015 BILBAO
T.: +34 946013732 | F.: +34 946013754
www.et.bs.ehu.es
2:43 p.m.
On Thu, 29 Nov 2007, Ignacio Diaz-Emparanza wrote:
El Thursday 29 November 2007 11:02:03 Franck Nadaud escribió:
> Hello listers ! greetings from Paris !
>
>> This would be brilliant.
>
> Sure it would be useful in some contexts. I am on the way to convert some
> old scripts I wrote in SPSS matrix processing language some (long) time
> ago. I plan to propose the Moran and Geary spatial correlation coefficients
> first. They are used as residuals mispecification tests when a regression
> model uses data related to a spatial context (States, counties, measure
> stations, etc...).
>
>> One of the thing planned for future versions is a function for
>> reading/writing matrices from/to csv files. Would this be useful to you?
>
> That would be exactly what is needed for the task. In fact most spatial
> analysis toolboxes / softwares proceed exactly that way around.
There is'nt actually a standard procedure for saving and reading matrices,
but Allin was working about this subject some months ago. You can read his
message to the gret-devel list in
http://lists.wfu.edu/pipermail/gretl-devel/2007-April/000326.html
Probably I should not recommend this tricky procedure to everyone, but ....
I agree this is very long-winded and cumbersome. In my local tree here
I've written two new functions to read/write matrices more easily, as in
matrix X = mread("foo.csv")
scalar errcode = mwrite(X, "bar.csv")
However, I won't commit them until Allin has seen them: they're full of
non-obvious choices, and probably of programming mistakes, so don't hold
your breath. Anyway, this is the direction that seems sensible to me. If
people come up with better ideas, they're most welcome.
Riccardo (Jack) Lucchetti
Dipartimento di Economia
Università Politecnica delle Marche
r.lucchetti(a)univpm.it
http://www.econ.univpm.it/lucchetti
5986
days inactive
5987
days old
4 comments
3 participants
participants (3)
-
Franck Nadaud -
Ignacio Diaz-Emparanza -
Riccardo (Jack) Lucchetti