Allow shell commands in gretl-git, linux server with out using the GUI
by leonardosuarezromero＠gmail.com
Does anyone know how to allow shell commands in Gretl-git built-based Linux server without using the GUI?
I'm using HANSL commands as backend services for a data-science project in a Debian-Linux server and it results to be that HANSL mixed with Bash-Shell Commands it can really do amazing things, I'm creating the project in a Supercomputer 504 teraflops. But when trying to configure the webserver based in shell-commands I can not allow the shell commands. In the documentation, explicitly says that for security reasons it can only be available using the GUI. But I was wondering if is it possible to allow the command by building the program in the ./configure command when installing in a Linux based server?
Kind regards
5 years
towards packages with R in the background
by Sven Schreiber
Hi,
at the conference we agreed that in the future contributed function
packages could use 'foreign' blocks that use R. (Just to be clear: the
big difference here is "contributed". A function package per se can
already use whatever is possible in hansl, for private use. But
currently it won't be accepted to the package server if it relies on a
foreign language.)
We also discussed that the package author must specify the version
requirement for R itself and then specify the needed R packages (along
with their own versions). I would like to propose a way to put this
information in the package's spec file, something like this:
First a list of needed ingredients:
R-packages = R glmnet
The idea is that first 'R' itself needs to be explicitly mentioned,
because it could be that no further R packages are needed. Also, it's
just one character :-)
Then the list of needed version requirements:
R-versions = 3.4.3 1.9-9
Obviously, the ordering here must match that in the 'R-packages' line.
As a refinement, several known-good versions might be specified in a
grouped item enclosed in curly braces:
R-versions = {3.4.3 3.4.4} 1.9-9
From what I understood of our discussion, gretl would not really check
the version requirements, but would use this information to print a
warning message at the 'include' stage.
Something like:
"This user-contributed gretl package requires a local R installation at
the path configured in gretl's preferences. (Needed version: 3.4.3, 3.4.4)
Required R packages: glmnet (version: 1.9-9)
The package will likely fail (with obscure error messages) if any of
those requirements are missing!"
Again, since gretl won't do any real checking, the needed changes do not
seem difficult, mainly a matter of documentation I guess.
Comments welcome.
thanks,
sven
5 years
String-valued series encoding
by Riccardo (Jack) Lucchetti
Hi all,
I've had a recurring problem with string-valued series, and I'm struggling
to find a solution.
Suppose you have two or more string-valued series that you get from a csv
or Stata file, and that they represent comparable variables, so they
contain the same strings. Currently, we encode string-valued series by
creating string arrays that get filled by occurrence; this, however,
impleis that there is no guarantee that the correspondence between
internal numerical values and strings is the same for the different
series. This makes it awkward to read the output from commands such as
freq or xtab.
Writing a script to correct for that has proven quite difficult, and what
I was able to come up with is VERY far from elegant. An example script
follows, and suggestions are much appreciated.
<hansl>
set verbose off
function series string_reorder(strings new, series x)
strings ss = strvals(x)
n = nelem(ss)
m = nelem(new)
series tmp = NA
loop i = 1 .. n --quiet
si = ss[i]
k = 0
loop j = 1 .. m --quiet
if si == new[j]
k = j
break
endif
endloop
if k>0 # found
tmp = (x == si) ? k : tmp
endif
endloop
return tmp
end function
clear
set verbose off
outfile "@dotdir/tmp.csv"
printf "var1,var2,var3\n"
printf "a,b,c\na,c,b\nb,b,b\nc,a,b\na,b,c\na,c,c"
end outfile
open "@dotdir/tmp.csv" --quiet
print var1 var2 --byobs
xtab var1 var2 # no good
# record encoding for var1
ss = strvals(var1)
# note: you can't just assign to var2
var2new = string_reorder(ss, var2)
stringify(var2new, ss)
delete var2
rename var2new var2
# values are the same, but the encoding is reordered
print var1 var2 --byobs
xtab var1 var2 # better
</hansl>
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
5 years, 1 month
'omit const' doesn't work
by Sven Schreiber
Hi,
consider this:
<hansl>
nulldata 10
series x = normal()
ols x const
stat = $coeff(const) / $stderr(const)
printf "Signif. of const: %g\n", 2* pvalue(t, $T-1, abs(stat))
omit const # error
</hansl>
This should work, no?
thanks
sven
5 years, 1 month
--panel-time option not working for all frequencies
by Artur Tarassow
Dear all,
it took me a while to understand that $setobs is only applicable for
panel data once the time-dimension of the panel data set is set using
the setobs command with the '--panel-time' option.
However, this is only seems to work for annual and quarterly data
currently. See the following example which returns empty series for
frequencies 5 and 7:
<hansl>
clear
set verbose off
open grunfeld.gdt -q
# WORKS
setobs 1 2000 --panel-time
series obsdate = $obsdate
print obsdate -o --range=1:20
# WORKS
setobs 4 2000:1 --panel-time
series obsdate = $obsdate
print obsdate -o --range=1:20
# DOESN'T WORK
setobs 5 2000:1 --panel-time
series obsdate = $obsdate
print obsdate -o --range=1:20
# DOESN'T WORK
setobs 7 2000:1 --panel-time
series obsdate = $obsdate
print obsdate -o --range=1:20
</hansl>
The following function is a workaround:
<hansl>
function series get_obsdate_for_panel (const string init_date "YYYY-MM-DD",
const int frequency[7]
"Frequency of time-dimension",
const series crossdim
"Series defining the cross-sectional dim.")
/* Construct a panel series of $obsdate.
THIS FUNCTION IS OBSOLETE FOR ANNUAL AND QUARTERLY FREQUENCIES ONLY.
SIMPLY USE:
<hansl>
open grunfeld.gdt -q
setobs 1 1935 --panel-time
series obsdate = $obsdate
</hansl>
*/
if $datatype != 3
funcerr "This function only works for panel data."
endif
string str_frequency = sprintf("%d", frequency)
scalar T = $pd
scalar N = $tmax/T
smpl crossdim == min(crossdim) --restrict
setobs @str_frequency @init_date --time-series
matrix obsmat = $obsdate
smpl full
matrix mat = obsmat * ones(1, N) # T by N
series obsdate = vec(mat)
return obsdate
end function
</hansl>
Best,
Artur
5 years, 1 month
the flatten() function
by Allin Cottrell
The flatten() function for arrays of matrices or strings has a couple
of enhancements in git.
Matrices: complex matrices are now handled correctly (but an array
containing a mixture of complex and real matrices is not accepted).
Strings: the second (optional boolean) argument to flatten can now be
used to separate the strings with spaces rather than newlines.
On strings, though, I have a comment on the feature request for the
new option (in the SF trackers). The text includes: "This would be
useful if the input array actually holds textual representations of
numbers, which can then be processed with sscanf."
An array of strings that represent numbers can be processed via sscanf
"as is", without flattening (see the help text for sscanf). And if the
array is flattened, sscanf will work equally well whether the strings
are on separate lines (you get a column vector) or space-separated on
a single line (you get a row vector).
<hansl>
strings S = defarray("1", "2", "3", "4")
matrix m
sscanf(S, "%m", m)
print m
sscanf(flatten(S), "%m", m)
print m
sscanf(flatten(S, 1), "%m", m)
print m
</hansl>
Allin
5 years, 1 month
bread: bug with zero-element matrix inside bundle
by Sven Schreiber
Hi,
I stumbled across the following bug in the MPI context, but it turns out
it doesn't have anything to do with MPI.
See:
<hansl>
bundle b = null
matrix m = zeros(2,0)
eval nelem(m)
err = mwrite(m, "mcheck.mat", 1)
print err
# re-load
matrix m2 = mread("mcheck.mat", 1) # works
print m2 # correct info ("is empty")
# bundle wrap
matrix b.m = m
err = bwrite(b, "bcheck.xml", 1) # seems to work
bundle b2 = bread("bcheck.xml", 1) # data error
eval $error
</hansl>
Seems the bread() function is not capable of handling the case of
cols="0" (or rows="0") inside the xml-represented bundle.
thanks
sven
5 years, 2 months
MPI and dataset context
by Sven Schreiber
Hi parallel processers,
I noticed there's one additional peculiarity about using MPI: Since it's
a different environment, the mpi block apparently doesn't get any
information about the currently active dataset. I have worked around
that problem in my application by putting in a 'nulldata ... --preserve'
at the mpi toplevel (after reading in the bundle with a contained list).
This seems to work, but here are still some questions or suggestions.
- We have a --send-functions option to mpi; wouldn't it make sense to
also have something like --send-data, which would resemble what we
already do for 'foreign'.
- Slightly more general than purely MPI-related: I noticed in the
gretl+MPI guide section 12 (ML estimation, illustration) it is explained
that some dummy 'nulldata' commands must be given just because the 'mle'
command formally requires it - although as explained there as well it
really isn't using any dataset in that case. This feels very awkward
and/or clumsy. Shouldn't the dataset requirement be restricted to those
commands which really need it?
thanks
sven
5 years, 2 months
Matrix Row-/ column names after selifc()/r()
by Artur Tarassow
Hi all,
it's cool that row or column names of matrices remain intact when
applying the msortby() function.
However, when applying row or column selection by means of selifr() or
selifc() destroys the attached labels of the respective dimension. See
below for examples.
This is a bit annoying for practical work. Would it be possible to make
sure that the labeling remains intact?
Best,
Artur
<hansl>
clear
set verbose off
matrix M = mnormal(3,2)
cnameset(M, defarray("C1", "C2"))
rnameset(M, defarray("R1", "R2", "R3"))
print M
# sorting does not affect row names
eval msortby(M,1)
eval msortby(M,2)
# select rows
matrix row_select = {1;1;1}
eval selifr(M, row_select) # row names still fine + col names fine
matrix row_select = {1;1;0}
eval selifr(M, row_select) # row names lost + col names fine
matrix row_select = {1;0;1}
eval selifr(M, row_select) # row names lost + col names fine
# select columns
matrix col_select = {1;1}'
eval selifc(M, col_select) # col names still fine + row names fine
matrix col_select = {1;0}'
eval selifc(M, col_select) # col names lost + row names fine
matrix col_select = {0;1}'
eval selifc(M, col_select) # col names lost + row names fine
</hansl>
5 years, 2 months
Power outage
by Riccardo (Jack) Lucchetti
* With apologies for cross-posting *
Hi everyone,
tomorrow morning there will be some work done on the power lines for the
building where our mailing list server is. Therefore, the mailing lists
are going to be down for some time; most likely, 7am - 1pm.
After that, everything should bo back to normal.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
5 years, 2 months
