3:49 p.m.
I was thinking that we might want to map what we call "discrete" series
into what R calls "factors". The idea is quite simple, and exemplified in
the script below. My question to the list is: is this a good idea? Is it
worth the coding effort (very small IMO)?
Comments welcome.
<hansl>
nulldata 50
cont1 = normal()
cont2 = normal()
disc1 = floor(uniform(1,5))
disc2 = floor(uniform(4,18))
discrete disc1 disc2
list D = dataset
loop foreach i D
if !isdiscrete($i)
D -= $i
endif
endloop
matrix mD = D
mwrite(mD, "discrete.mat", 1)
foreign language=R --send-data
D <- gretl.loadmat("discrete.mat");
for (i in D) {gretldata[,i] <- as.factor(gretldata[,i])};
summary(gretldata);
end foreign
</hansl>
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
9:20 p.m.
On Wed, 13 Sep 2017, Riccardo (Jack) Lucchetti wrote:
I was thinking that we might want to map what we call
"discrete"
series into what R calls "factors". The idea is quite simple, and
exemplified in the script below. My question to the list is: is
this a good idea? Is it worth the coding effort (very small IMO)?
Comments welcome.
<hansl>
nulldata 50
cont1 = normal()
cont2 = normal()
disc1 = floor(uniform(1,5))
disc2 = floor(uniform(4,18))
discrete disc1 disc2
list D = dataset
loop foreach i D
if !isdiscrete($i)
D -= $i
endif
endloop
matrix mD = D
mwrite(mD, "discrete.mat", 1)
foreign language=R --send-data
D <- gretl.loadmat("discrete.mat");
for (i in D) {gretldata[,i] <- as.factor(gretldata[,i])};
summary(gretldata);
end foreign
</hansl>
Not sure about this, but my initial reaction is that it may be
assuming too much about our "discrete" series.
In R, isn't a "factor" a variable that (in gretl parlance) has to be
"dummified" before use in regression? That is, an arbitrary
encoding of a qualitative characteristic?
If so, then I think the above is wrong, since a gretl-discrete
series could be a perfectly valid (albeit quantized) quantitative
variable; for example, years of education or number of bedrooms.
But If I'm wrong about what a "factor" is to R, my objection may
fall.
Allin
12:57 a.m.
On Wed, 13 Sep 2017, Allin Cottrell wrote:
On Wed, 13 Sep 2017, Riccardo (Jack) Lucchetti wrote:
> I was thinking that we might want to map what we call "discrete" series
> into what R calls "factors". The idea is quite simple, and exemplified in
> the script below. My question to the list is: is this a good idea? Is it
> worth the coding effort (very small IMO)?
>
> Comments welcome.
>
> <hansl>
> nulldata 50
>
> cont1 = normal()
> cont2 = normal()
> disc1 = floor(uniform(1,5))
> disc2 = floor(uniform(4,18))
>
> discrete disc1 disc2
>
> list D = dataset
> loop foreach i D
> if !isdiscrete($i)
> D -= $i
> endif
> endloop
>
> matrix mD = D
> mwrite(mD, "discrete.mat", 1)
>
> foreign language=R --send-data
> D <- gretl.loadmat("discrete.mat");
> for (i in D) {gretldata[,i] <- as.factor(gretldata[,i])};
> summary(gretldata);
> end foreign
> </hansl>
Not sure about this, but my initial reaction is that it may be assuming too
much about our "discrete" series.
In R, isn't a "factor" a variable that (in gretl parlance) has to be
"dummified" before use in regression? That is, an arbitrary encoding of a
qualitative characteristic?
If so, then I think the above is wrong, since a gretl-discrete series could
be a perfectly valid (albeit quantized) quantitative variable; for example,
years of education or number of bedrooms.
But If I'm wrong about what a "factor" is to R, my objection may fall.
Sorry, I should have added: we now have the facility, under the
"setinfo" command, of marking a series as "coded". And when we write
a "coded" series as CSV we quote the numerical values, in response
to which R automatically treats the series as a "factor". So I think
we already have what you're aiming at here.
Allin
6:10 a.m.
On Wed, 13 Sep 2017, Allin Cottrell wrote:
> Not sure about this, but my initial reaction is that it may be
assuming too
> much about our "discrete" series.
>
> In R, isn't a "factor" a variable that (in gretl parlance) has to be
> "dummified" before use in regression? That is, an arbitrary encoding of a
> qualitative characteristic?
Yes, you're right.
> If so, then I think the above is wrong, since a gretl-discrete
series could
> be a perfectly valid (albeit quantized) quantitative variable; for example,
> years of education or number of bedrooms.
>
> But If I'm wrong about what a "factor" is to R, my objection may fall.
Sorry, I should have added: we now have the facility, under the "setinfo"
command, of marking a series as "coded". And when we write a "coded"
series
as CSV we quote the numerical values, in response to which R automatically
treats the series as a "factor". So I think we already have what you're
aiming at here.
I agree that the mapping to R's factors is much more accurate if we used
the "coded" bit. However, R doesn't seem to make this distinction
automagically for integer-valued coded strings. Example:
<hansl>
nulldata 50
cont1 = normal()
disc1 = floor(uniform(1,5))
disc2 = floor(uniform(4,18))
stringify(disc1, defarray("a", "b", "c", "d")) #
string-valued series
list D = disc1 disc2
loop foreach i D
setinfo $i --coded
endloop
foreign language=R --send-data
summary(gretldata);
is.factor(gretldata$disc1);
is.factor(gretldata$disc2);
end foreign
</hansl>
Perhaps we could force R to treat variables as factors via an additional
option to foreign, something like
foreign language=R --send-data --as-factors=X
where X is a list.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:07 a.m.
I nave no strong opinion, but
dealing with factors in R is
very simple:
foreign language=R
set.seed(13)
y <- rnorm(10)
x <- sample(3,10,replace=T)
lm(y~x)
lm(y~factor(x))
xch <- as.character(x)
print(xch)
lm(y~xch)
end foreign
As for me, the most tedious thing
with Main/Tools/Start GNU R
are time series
In this case class(gretldata)->"mts"
That's why I never use Tools/Start GNU R
with time series: instead I run the following
R script from gretl
<R
# choose interactive R session
if (any(class(gretldata)!="data.frame"))
{mts2frame2<-function(x)
{ddd<-data.frame(x);
if (length(class(x))>1)
{m<-ncol(x);for (i in 1:m)
ddd[,i]<-ts(ddd[,i],start=start(x),frequency=frequency(x))}
else
ddd[,1]<-ts(ddd[,1],start=start(x),frequency=frequency(x));
ddd};
gretldata<-mts2frame2(gretldata)}
# make native gretl variables names searcheable
attach(gretldata)
R>
In this case gretldata becomes a data.frame
with its columns retaining dates and frequencies
Oleh
14 вересня 2017, 00:20:54, від "Allin Cottrell" <cottrell(a)wfu.edu>:
> On Wed, 13 Sep 2017, Riccardo (Jack) Lucchetti wrote:
> I was thinking that we might want to map what we call "discrete"
> series into what R calls "factors". The idea is quite simple, and
> exemplified in the script below. My question to the list is: is
> this a good idea? Is it worth the coding effort (very small IMO)?
>
> Comments welcome.
>
> <hansl>
> nulldata 50
>
> cont1 = normal()
> cont2 = normal()
> disc1 = floor(uniform(1,5))
> disc2 = floor(uniform(4,18))
>
> discrete disc1 disc2
>
> list D = dataset
> loop foreach i D
> if !isdiscrete($i)
> D -= $i
> endif
> endloop
>
> matrix mD = D
> mwrite(mD, "discrete.mat", 1)
>
> foreign language=R --send-data
> D <- gretl.loadmat("discrete.mat");
> for (i in D) {gretldata[,i] <- as.factor(gretldata[,i])};
> summary(gretldata);
> end foreign
> </hansl>
Not sure about this, but my initial reaction is that it may be
assuming too much about our "discrete" series.
In R, isn't a "factor" a variable that (in gretl parlance) has to be
"dummified" before use in regression? That is, an arbitrary
encoding of a qualitative characteristic?
If so, then I think the above is wrong, since a gretl-discrete
series could be a perfectly valid (albeit quantized) quantitative
variable; for example, years of education or number of bedrooms.
But If I'm wrong about what a "factor" is to R, my objection may
fall.
Allin
_______________________________________________
Gretl-devel mailing list
Gretl-devel@lists.wfu.eduhttp://lists.wfu.edu/mailman/listinfo/gretl-devel
12:06 a.m.
On Thu, 14 Sep 2017, Riccardo (Jack) Lucchetti wrote:
On Wed, 13 Sep 2017, Allin Cottrell wrote:
>> Not sure about this, but my initial reaction is that it may be assuming
>> too much about our "discrete" series.
>>
>> In R, isn't a "factor" a variable that (in gretl parlance) has to
be
>> "dummified" before use in regression? That is, an arbitrary encoding
of a
>> qualitative characteristic?
Yes, you're right.
>> If so, then I think the above is wrong, since a gretl-discrete series
>> could be a perfectly valid (albeit quantized) quantitative variable; for
>> example, years of education or number of bedrooms.
>>
>> But If I'm wrong about what a "factor" is to R, my objection may
fall.
>
> Sorry, I should have added: we now have the facility, under the "setinfo"
> command, of marking a series as "coded". And when we write a
"coded" series
> as CSV we quote the numerical values, in response to which R automatically
> treats the series as a "factor". So I think we already have what you're
> aiming at here.
I agree that the mapping to R's factors is much more accurate if we used the
"coded" bit. However, R doesn't seem to make this distinction automagically
for integer-valued coded strings. Example:
<hansl>
nulldata 50
cont1 = normal()
disc1 = floor(uniform(1,5))
disc2 = floor(uniform(4,18))
stringify(disc1, defarray("a", "b", "c", "d")) #
string-valued series
list D = disc1 disc2
loop foreach i D
setinfo $i --coded
endloop
foreign language=R --send-data
summary(gretldata);
is.factor(gretldata$disc1);
is.factor(gretldata$disc2);
end foreign
</hansl>
Hmm, I see what you mean. Not sure where I got the idea that "quoted
in CSV" means factor to R, but apparently it's not true in general.
Perhaps we could force R to treat variables as factors via an
additional
option to foreign, something like
foreign language=R --send-data --as-factors=X
where X is a list.
In git there's now something more automated than that: we have a
variant of what you sketched in
http://lists.wfu.edu/pipermail/gretl-devel/2017-September/007916.html
whereby we send R a matrix that identifies any "coded" series in
gretl as factors for R. (We handle the case where the data passed to
R are a subset of the full dataset, as in --send-data=Rlist.)
Allin
1:08 a.m.
On Thu, 14 Sep 2017, oleg_komashko(a)ukr.net wrote:
[...]
As for me, the most tedious thing
with Main/Tools/Start GNU R
are time series
In this case class(gretldata)->"mts"
That's why I never use Tools/Start GNU R
with time series: instead I run the following
R script from gretl
<R
# choose interactive R session
if (any(class(gretldata)!="data.frame"))
{mts2frame2<-function(x)
{ddd<-data.frame(x);
if (length(class(x))>1)
{m<-ncol(x);for (i in 1:m)
ddd[,i]<-ts(ddd[,i],start=start(x),frequency=frequency(x))}
else
ddd[,1]<-ts(ddd[,1],start=start(x),frequency=frequency(x));
ddd};
gretldata<-mts2frame2(gretldata)}
# make native gretl variables names searcheable
attach(gretldata)
R>
Is that the general opinion? That is, that R's multiple time series
object ("mts") is of no use and we're better creating a data frame
and copying the time-series attributes across per series.
If so, we could easily revise our R-data exporter to do just that
for time series datasets.
Allin
5:09 a.m.
On Thu, 14 Sep 2017, Allin Cottrell wrote:
On Thu, 14 Sep 2017, oleg_komashko(a)ukr.net wrote:
[...]
> As for me, the most tedious thing
> with Main/Tools/Start GNU R
> are time series
> In this case class(gretldata)->"mts"
>
> That's why I never use Tools/Start GNU R
> with time series: instead I run the following
> R script from gretl
> <R
> # choose interactive R session
> if (any(class(gretldata)!="data.frame"))
> {mts2frame2<-function(x)
> {ddd<-data.frame(x);
> if (length(class(x))>1)
> {m<-ncol(x);for (i in 1:m)
> ddd[,i]<-ts(ddd[,i],start=start(x),frequency=frequency(x))}
> else
> ddd[,1]<-ts(ddd[,1],start=start(x),frequency=frequency(x));
> ddd};
> gretldata<-mts2frame2(gretldata)}
>
> # make native gretl variables names searcheable
> attach(gretldata)
> R>
Is that the general opinion? That is, that R's multiple time series object
("mts") is of no use and we're better creating a data frame and copying the
time-series attributes across per series.
If so, we could easily revise our R-data exporter to do just that for time
series datasets.
Personally, I have a totally different opinion from Oleh's. I like the
fact that we ship time-series data into R as a ts(), as opposed to a
generic data frame.
But, that said, I'm not a heavy R user, so I'd rather listen to more
informed people than myself.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:31 a.m.
Am 15.09.2017 um 07:09 schrieb Riccardo (Jack) Lucchetti:
On Thu, 14 Sep 2017, Allin Cottrell wrote:
> Is that the general opinion? That is, that R's multiple time
series
> object ("mts") is of no use and we're better creating a data frame and
> copying the time-series attributes across per series.
Personally, I have a totally different opinion from Oleh's. I
like the
fact that we ship time-series data into R as a ts(), as opposed to a
generic data frame.
But, that said, I'm not a heavy R user, so I'd rather listen to more
informed people than myself.
Yes, Oleh please explain what the advantages of your method are over
ts/mts or whatever it's called in R.
thanks,
sven
11:54 a.m.
On Fri, 15 Sep 2017, Sven Schreiber wrote:
Am 15.09.2017 um 07:09 schrieb Riccardo (Jack) Lucchetti:
> On Thu, 14 Sep 2017, Allin Cottrell wrote:
>> Is that the general opinion? That is, that R's multiple time series object
>> ("mts") is of no use and we're better creating a data frame and
copying
>> the time-series attributes across per series.
> Personally, I have a totally different opinion from Oleh's. I like the fact
> that we ship time-series data into R as a ts(), as opposed to a generic
> data frame.
>
> But, that said, I'm not a heavy R user, so I'd rather listen to more
> informed people than myself.
Yes, Oleh please explain what the advantages of your method are over ts/mts
or whatever it's called in R.
Well, one point in favor of Oleh's approach is that you can attach() a
data frame but not an "mts". So for example if you send our
denmark.gdt to R in Oleh's way you can refer to "LRM", "LRY" etc
by
name. As things stand, you can't. And you can't even say
"gretldata$LRM" ("$ operator is invalid for atomic vectors") although
you can refer to LRM as gretldata[,1].
You can turn an mts into a data frame if you like:
dfr <- data.frame(gretldata)
attach(dfr)
and then refer to "LRM", but annoyingly that loses the time-series
information. Oleh's trick is to make a data frame but then turn each
of its columns into a ts. Seems to me this is a workaround for
something that's kinda broken in R, but it works.
Allin
7:38 p.m.
On Thu, 14 Sep 2017, Allin Cottrell wrote:
On Thu, 14 Sep 2017, oleg_komashko(a)ukr.net wrote:
[...]
> As for me, the most tedious thing
> with Main/Tools/Start GNU R
> are time series
> In this case class(gretldata)->"mts"
[W]e could easily revise our R-data exporter [...]
Now in git and snapshots I've made an option available. Under the
"foreign" command the option is --frame; under "/Tools/Start GNU R"
it's a little dialog box with the choices to send our data (a) as a
multiple time series object (what we already do) or as a data frame
(as per Oleh, with columns set as "ts").
These things only apply if the current gretl dataset is time series.
Illustration:
<hansl>
open data9-7 --quiet
list L = QNC INCOME
# the default method
foreign language=R --send-data=L
gretldata
gretldata[,1]
# "QNC" would not work here
end foreign
# Oleh's preferred method
foreign language=R --send-data=L --frame
gretldata
QNC
end foreign
</hansl>
Allin
2408
days inactive
2410
days old
10 comments
4 participants
participants (4)
-
Allin Cottrell -
oleg_komashko＠ukr.net -
Riccardo (Jack) Lucchetti -
Sven Schreiber