Hi All,
This is my first adaptation of Allin's test package.
Right now I'm very pressed for time, but I'm putting my test rig at
http://ricardo.ecn.wfu.edu/~cottrell/testing/test-gretl.tar.xz
so anyone can take a look. It exercises almost 20000 scripts; the
testing mechanism utilizes "make" and shell scripts. Each directory
contains "output" and "newout" subdirectories. The basic idea is that
"make" in a given directory will populate the "newout" directory with
output files, then run a "diff" on newout vs "output" (besides reporting
any failures). So "output" is supposed to contain "known good" results.
You can run everything by typing "make test-all" in the top-level
directory. This may take a while; it will produce a composite diff of
all new output against all previous output.
Anyone running this on their own system should first go into the "bin"
directory (under "test-gretl") and edit the file named "sitevars".
This setup still must be done/checked. Path to gretl and libgretl. After this, it should be fine for any (Linux) user.
Attached is a diff of my files vs Allin's.
(There are obvious differences in R and Octave because of missing libraries, and missing files, and applications)
My system is Linux CentOS 7.
I don't know if the "baseline" outputs do match the released 2016a (I suspect not).
These are the changes I have done:
Force running Gretl in English.
After Gretl "newout" has been created those files are edited, so that all user ($HOME and test-gretl) paths are replaced to simulate "output" original files.
TODO: Remove inconsistent paths, on Allin's original "output" files preparation (make new).
Try to not compare with "output", when applications are not installed (Stata, OX, ...)
Obtain system info (kind of benchmarking, or performance factoring)
Should we also do the NIST tests?
There is a test that creates two Gnuplot plots, maybe they should have output to PNG.
When I tested in Portuguese, there were warnings on strings max length (need tests for all languages).
---- EOT ----