Hi All,

This is my first adaptation of Allin's test package.

Right now I'm very pressed for time, but I'm putting my test rig at

http://ricardo.ecn.wfu.edu/~cottrell/testing/test-gretl.tar.xz

so anyone can take a look. It exercises almost 20000 scripts; the testing mechanism utilizes "make" and shell scripts. Each directory contains "output" and "newout" subdirectories. The basic idea is that "make" in a given directory will populate the "newout" directory with output files, then run a "diff" on newout vs "output" (besides reporting any failures). So "output" is supposed to contain "known good" results.

You can run everything by typing "make test-all" in the top-level directory. This may take a while; it will produce a composite diff of all new output against all previous output.

Anyone running this on their own system should first go into the "bin" directory (under "test-gretl") and edit the file named "sitevars".

This setup still must be done/checked. Path to gretl and libgretl. After this, it should be fine for any (Linux) user.

The test-gretl.zip (17MB) is https://drive.google.com/open?id=0Bx8T-KbkRZold0pkTW1rMDZBZG8

Attached is a diff of my files vs Allin's.

I also attached here analysis of 2016a and development versions against the original tests from Allin's package.
All these files are at this folder https://drive.google.com/folderview?id=0Bx8T-KbkRZolWFE5MWxGNm5LLUU&usp=sharing

(There are obvious differences in R and Octave because of missing libraries, and missing files, and applications)

My system is Linux CentOS 7.

I don't know if the "baseline" outputs do match the released 2016a (I suspect not).

These are the changes I have done:

Force running Gretl in English.

After Gretl "newout" has been created those files are edited, so that all user ($HOME and test-gretl) paths are replaced to simulate "output" original files.

TODO: Remove inconsistent paths, on Allin's original "output" files preparation (make new).

Try to not compare with "output", when applications are not installed (Stata, OX, ...)

Obtain system info (kind of benchmarking, or performance factoring)

Should we also do the NIST tests?

There is a test that creates two Gnuplot plots, maybe they should have output to PNG.

When I tested in Portuguese, there were warnings on strings max length (need tests for all languages).

Have a nice Monday! :)

Hélio