Hi All,
This is my first adaptation of Allin's test package.
Right now I'm very pressed for time, but I'm putting my test
rig at
http://ricardo.ecn.wfu.edu/~cottrell/testing/test-gretl.tar.xz
so anyone can take a look. It exercises almost 20000 scripts; the testing
mechanism utilizes "make" and shell scripts. Each directory contains
"output" and "newout" subdirectories. The basic idea is that
"make" in a
given directory will populate the "newout" directory with output files,
then run a "diff" on newout vs "output" (besides reporting any
failures).
So "output" is supposed to contain "known good" results.
You can run everything by typing "make test-all" in the top-level
directory. This may take a while; it will produce a composite diff of all
new output against all previous output.
Anyone running this on their own system should first go into the "bin"
directory (under "test-gretl") and edit the file named "sitevars".
This setup still must be done/checked. Path to gretl and libgretl. After
this, it should be fine for any (Linux) user.
The test-gretl.zip (17MB) is
https://drive.google.com/open?id=0Bx8T-KbkRZold0pkTW1rMDZBZG8
Attached is a diff of my files vs Allin's.
I also attached here analysis of 2016a and development versions against the
original tests from Allin's package.
All these files are at this folder
https://drive.google.com/folderview?id=0Bx8T-KbkRZolWFE5MWxGNm5LLUU&u...
(There are obvious differences in R and Octave because of missing
libraries, and missing files, and applications)
My system is Linux CentOS 7.
I don't know if the "baseline" outputs do match the released 2016a (I
suspect not).
These are the changes I have done:
Force running Gretl in English.
After Gretl "newout" has been created those files are edited, so that all
user ($HOME and test-gretl) paths are replaced to simulate "output"
original files.
TODO: Remove inconsistent paths, on Allin's original "output" files
preparation (make new).
Try to not compare with "output", when applications are not
installed (Stata, OX, ...)
Obtain system info (kind of benchmarking, or performance
factoring)
Should we also do the NIST tests?
There is a test that creates two Gnuplot plots, maybe they
should have output to PNG.
When I tested in Portuguese, there were warnings on strings max
length (need tests for all languages).
Have a nice Monday! :)
Hélio