Allow shell commands in gretl-git, linux server with out using the GUI
by leonardosuarezromero＠gmail.com
Does anyone know how to allow shell commands in Gretl-git built-based Linux server without using the GUI?
I'm using HANSL commands as backend services for a data-science project in a Debian-Linux server and it results to be that HANSL mixed with Bash-Shell Commands it can really do amazing things, I'm creating the project in a Supercomputer 504 teraflops. But when trying to configure the webserver based in shell-commands I can not allow the shell commands. In the documentation, explicitly says that for security reasons it can only be available using the GUI. But I was wondering if is it possible to allow the command by building the program in the ./configure command when installing in a Linux based server?
Kind regards
3 years, 10 months
feval doesn't work with default argument
by Sven Schreiber
Hi,
I've noticed that 'feval' always requires the explicit passing of an
argument to the specified function, as in 'feval("myfunc", 1)'. So the
following trivial example fails:
<hansl>
function scalar myfunc(int in[1])
return in
end function
result = feval("myfunc") # intended use: default args
</hansl>
Certainly no big deal, but wouldn't that be a valid use of the 'feval'
apparatus? BTW, the current behavior is implicitly documented alright
("one or more further args needed").
thanks
sven
3 years, 11 months
towards packages with R in the background
by Sven Schreiber
Hi,
at the conference we agreed that in the future contributed function
packages could use 'foreign' blocks that use R. (Just to be clear: the
big difference here is "contributed". A function package per se can
already use whatever is possible in hansl, for private use. But
currently it won't be accepted to the package server if it relies on a
foreign language.)
We also discussed that the package author must specify the version
requirement for R itself and then specify the needed R packages (along
with their own versions). I would like to propose a way to put this
information in the package's spec file, something like this:
First a list of needed ingredients:
R-packages = R glmnet
The idea is that first 'R' itself needs to be explicitly mentioned,
because it could be that no further R packages are needed. Also, it's
just one character :-)
Then the list of needed version requirements:
R-versions = 3.4.3 1.9-9
Obviously, the ordering here must match that in the 'R-packages' line.
As a refinement, several known-good versions might be specified in a
grouped item enclosed in curly braces:
R-versions = {3.4.3 3.4.4} 1.9-9
From what I understood of our discussion, gretl would not really check
the version requirements, but would use this information to print a
warning message at the 'include' stage.
Something like:
"This user-contributed gretl package requires a local R installation at
the path configured in gretl's preferences. (Needed version: 3.4.3, 3.4.4)
Required R packages: glmnet (version: 1.9-9)
The package will likely fail (with obscure error messages) if any of
those requirements are missing!"
Again, since gretl won't do any real checking, the needed changes do not
seem difficult, mainly a matter of documentation I guess.
Comments welcome.
thanks,
sven
3 years, 11 months
Editor search results sometimes outside of visible window area
by Sven Schreiber
Hi,
this problem is not new, I have reported it before on Windows. Sometimes
you need to scroll up or down by one (or two?) lines to see the
highlighted match.
But I've recently also observed it on Linux Mint (with a self-compiled
git version from some weeks ago I think), so I thought I'd raise it
again. It is still a bit hard to pinpoint when exactly it happens, though.
cheers
sven
3 years, 11 months
SIGSEGV - dbnomics
by Marcin Błażejowski
Hi,
simple script:
<hansl>
set verbose off
open dbnomics
string KRAJE = "AT BE BG CY CZ DE DK ES EE FI FR GB GR HR HU IE IT LT LU
LV MT NL PL PT RO SK SI SE EA12 EA19 EU15 EU28"
loop foreach KR @KRAJE --quiet
data Eurostat/irt_lt_mcby_q/Q.MCBY.$KR
endloop
</hansl>
and we get SIGSEGV.
GDB output:
-------------------------------
Thread 1 "gretl_x11" received signal SIGSEGV, Segmentation fault.
write_dbn_csv (v=0x0, fp=0x5555557bb200, ds=0x0, T=<optimized out>,
S=0x55555613a150) at ../lib/src/dbnread.c:65
65 if (na(v->val[i])) {
(gdb)
-------------------------------
Marcin
--
Marcin Błażejowski
3 years, 11 months
SIGSEGV
by Marcin Błażejowski
Hi,
simple script, which works on older gretl versions, crushes on current
git. GDB says:
----------------------------------
Thread 1 "gretl_x11" received signal SIGSEGV, Segmentation fault.
0x00007ffff7444ed4 in make_dollar_substitutions
(str=str@entry=0x7ffffffdbc50 "$ZM_ECM_list", maxlen=maxlen@entry=32,
loop=loop@entry=0x555555887570, dset=dset@entry=0x55555580daa0,
subst=subst@entry=0x0, opt=opt@entry=OPT_T)
at ../lib/src/monte_carlo.c:3035
3035 *subst = 0;
----------------------------------
Marcin
--
Marcin Błażejowski
3 years, 11 months
MPI data transfer
by Allin Cottrell
I've been working recently on bumping up the efficiency of MPI data
transfer.
To date transfer of matrices has been pretty fast, going directly
via memory, but bundles have been a lot slower: we took the approach
of serializing the bundle to XML at the source end, passing the XML
bytes, then deserializing at the destination end. This has been
quite reliable (modulo a few bugs that Marcin has brought to light!)
but it's inherently clunky.
Now in git I've arranged for mpibcast, mpisend and mpirecv to work
directly for bundles -- and as a by-product, enabled MPI transfer
for arrays. I've done a fair amount of testing of "direct" mpibcast
of bundles, and that's currently the default in git. The default for
mpisend/mpirecv is still the old XML method (since I just coded the
new variant today and it's barely tested) but if you're building
gretl yourself you can try out the new variants thus: look for the
definition of ALT_BUNSEND in lib/src/gretl_mpi.c and change it from
0 to 1.
(I should mention, none of the new stuff is in the Windows or Mac
snapshots just yet.)
I'll do more testing myself, but if anyone else wants to join in,
please do. My results so far are very encouraging. Here's some
comparative output from a script that transfers a bundle holding an
array of 20 big matrices:
old XML-based version:
gretlmpi 2019d-git
Current session: 2019-10-17 15:00
have 20 matrices
bcast time: 1.63798s
got 20 matrices
send/recv time: 1.59941s
diff_arrays gave 8.88178e-16
new direct version:
gretlmpi 2019d-git
Current session: 2019-10-17 15:01
have 20 matrices
bcast time: 0.0132543s
got 20 matrices
send/recv time: 0.0054283s
diff_arrays gave 0
(The non-zero "diff_arrays" value for the XML method is a trivial
result of the round-trip serialization and deserialization, but it's
nice to see it go to true zero under the direct method.)
Here's the script in question:
<hansl>
set verbose off
function scalar diff_arrays (const matrices M1, const matrices M2)
scalar maxdiff = 0
loop i=1..nelem(M1) -q
scalar d = max(abs(M1[i] - M2[i]))
if d > maxdiff
maxdiff = d
endif
endloop
return maxdiff
end function
mpi --send-functions
bundle b
if $mpirank == 0
matrices b.MM = array(20)
loop i=1..nelem(b.MM) -q
b.MM[i] = mnormal(1000,100)
endloop
printf "have %d matrices\n", nelem(b.MM)
set stopwatch
endif
mpibcast(&b)
if $mpirank == 0
printf "bcast time: %gs\n", $stopwatch
endif
set stopwatch
if $mpirank == 1
printf "got %d matrices\n", nelem(b.MM)
mpisend(b, 0)
elif $mpirank == 0
c = mpirecv(1)
printf "send/recv time: %gs\n", $stopwatch
scalar d = diff_arrays(b.MM, c.MM)
printf "diff_arrays gave %g\n", d
endif
end mpi --np=4
</hansl>
Allin
3 years, 11 months
Effective testing OpenMP in algebra operations
by Marcin Błażejowski
Hi,
I'd like to compare speed of algebra operations when gretl uses
OpenBlas+openmp or OpenBlas+pthreads or Atlas and I'm looking for a
simple and effective script for such comparison.
9 years ago Alling posted 'matmult_test.inp', is this code sufficient
for my task?
Marcin
--
Marcin Błażejowski
3 years, 11 months
Speed of matrix "block" operations
by Marcin Błażejowski
Hi,
some time ago I found an interesting disscusion on code optimisiation in
gcc with '-avx2 ' flag in case of copying blocks of matrixes. So, I
wrote a simple script and I got the following results:
? set verbose off
10,586272
9,690394
10,030351
10,335968
8,9853768
Difference 1: 0,895878
Difference 2: -0,305616
I have to say that I'm little surprised. Although I would expect
difference between the first and the second loop ('Difference 1:
0,895878'), I wouldn't expect any difference between timings of loops
three and four ('Difference 2: -0,305616'). And, after all, why the
difference between the last loop and the other loops are so big?
Marcin
<hansl>
set verbose off
clear
scalar ROW = 10000
scalar COL = 20
scalar BLOCK_size = 2000
scalar BLOCK_start = 115
scalar LOOP = 600
matrix A = mnormal(ROW, COL)
matrix B1 = zeros(ROW, COL)
matrix B2 = B1
matrix B3 = B1
matrix B4 = B1
matrix B5 = B1
set stopwatch
loop LOOP --quiet
loop i=1..BLOCK_size --quiet
loop j=1..COL --quiet
B1[BLOCK_start - 1 + i, j] = A[BLOCK_start - 1 + i, j]
endloop
endloop
endloop
est1 = $stopwatch
eval est1
set stopwatch
loop LOOP --quiet
loop i=0..BLOCK_size-1 --quiet
loop j=1..COL --quiet
B2[BLOCK_start + i, j] = A[BLOCK_start + i, j]
endloop
endloop
endloop
est2 = $stopwatch
eval est2
loop LOOP --quiet
loop i=1..BLOCK_size --quiet
loop j=1..COL --quiet
tmp1 = BLOCK_start - 1 + i
B3[tmp1, j] = A[tmp1, j]
endloop
endloop
endloop
est3 = $stopwatch
eval est3
set stopwatch
loop LOOP --quiet
loop i=0..BLOCK_size-1 --quiet
loop j=1..COL --quiet
tmp2 = BLOCK_start + i
B4[tmp2, j] = A[tmp2, j]
endloop
endloop
endloop
est4 = $stopwatch
eval est4
loop LOOP --quiet
loop i=BLOCK_start..BLOCK_start-1+BLOCK_size --quiet
loop j=1..COL --quiet
B5[i, j] = A[i, j]
endloop
endloop
endloop
eval $stopwatch
printf "Difference 1: %f\n", est1 - est2
printf "Difference 2: %f\n", est3 - est4
</hansl>
--
Marcin Błażejowski
3 years, 11 months
