12:59 p.m.
Hi all,
this week I've found the following Julia page:
https://julialang.org/benchmarks/
The authors compare several software packages regarding their
performance for different tasks. I am missing gretl there! :-)
What do you think about sharing the work such that 1-2 people translate
some task/code to hansl. Once we have collected all replication files,
we send it to those Julia guys and ask for also getting compared.
This might be a good opportunity to get (better) known in the crowd.
Best,
Artur
2:33 p.m.
Am 25.02.2018 um 13:59 schrieb Artur Tarassow:
Hi all,
this week I've found the following Julia page:
https://julialang.org/benchmarks/
The authors compare several software packages regarding their
performance for different tasks. I am missing gretl there! :-)
What do you think about sharing the work such that 1-2 people translate
some task/code to hansl. Once we have collected all replication files,
we send it to those Julia guys and ask for also getting compared.
Given that slower things like pure Python, R or Octave are also on
there, yes why not. Perhaps we could start by putting together a
separate webpage (linked from gretl's start page) with the analogous
gretl/hansl numbers?
cheers,
sven
4:01 p.m.
On Sun, 25 Feb 2018, Sven Schreiber wrote:
Am 25.02.2018 um 13:59 schrieb Artur Tarassow:
> Hi all,
>
> this week I've found the following Julia page:
>
> https://julialang.org/benchmarks/
>
> The authors compare several software packages regarding their performance
> for different tasks. I am missing gretl there! :-)
>
> What do you think about sharing the work such that 1-2 people translate
> some task/code to hansl. Once we have collected all replication files, we
> send it to those Julia guys and ask for also getting compared.
Given that slower things like pure Python, R or Octave are also on there, yes
why not. Perhaps we could start by putting together a separate webpage
(linked from gretl's start page) with the analogous gretl/hansl numbers?
Here's a start (attached).
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
6:16 p.m.
Am 25.02.2018 um 17:01 schrieb Riccardo (Jack) Lucchetti:
On Sun, 25 Feb 2018, Sven Schreiber wrote:
> Am 25.02.2018 um 13:59 schrieb Artur Tarassow:
>> Hi all,
>>
>> this week I've found the following Julia page:
>>
>> https://julialang.org/benchmarks/
>>
>> The authors compare several software packages regarding their
>> performance for different tasks. I am missing gretl there! :-)
>>
>> What do you think about sharing the work such that 1-2 people
>> translate some task/code to hansl. Once we have collected all
>> replication files, we send it to those Julia guys and ask for also
>> getting compared.
>
> Given that slower things like pure Python, R or Octave are also on
> there, yes why not. Perhaps we could start by putting together a
> separate webpage (linked from gretl's start page) with the analogous
> gretl/hansl numbers?
Here's a start (attached).
Thanks for the work, Jack. I've added the remaining ones. However, I am
not sure about the last printfd(), and also added a few FIXME flags.
Maybe somebody could check this.
Best,
Artur
6:42 p.m.
On Sun, 25 Feb 2018, Artur Tarassow wrote:
Am 25.02.2018 um 17:01 schrieb Riccardo (Jack) Lucchetti:
> On Sun, 25 Feb 2018, Sven Schreiber wrote:
>
>> Am 25.02.2018 um 13:59 schrieb Artur Tarassow:
>>> Hi all,
>>>
>>> this week I've found the following Julia page:
>>>
>>> https://julialang.org/benchmarks/
>>>
>>> The authors compare several software packages regarding their performance
>>> for different tasks. I am missing gretl there! :-)
>>>
>>> What do you think about sharing the work such that 1-2 people translate
>>> some task/code to hansl. Once we have collected all replication files, we
>>> send it to those Julia guys and ask for also getting compared.
>>
>> Given that slower things like pure Python, R or Octave are also on there,
>> yes why not. Perhaps we could start by putting together a separate webpage
>> (linked from gretl's start page) with the analogous gretl/hansl numbers?
>
> Here's a start (attached).
Thanks for the work, Jack. I've added the remaining ones. However, I am not
sure about the last printfd(), and also added a few FIXME flags. Maybe
somebody could check this.
Thanks, Artur; btw, there was a bug in the Mandelbrot part. I'll try to
get some more work done asap.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:25 p.m.
On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
Thanks, Artur; btw, there was a bug in the Mandelbrot part. I'll
try to get
some more work done asap.
This should be it.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
10:40 a.m.
Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
> Thanks, Artur; btw, there was a bug in the Mandelbrot part. I'll try
> to get some more work done asap.
This should be it.
Thanks, Jack. I am just trying to find out who the responsible person
for the Julia project is. Any idea?
It seems that the scripts for different software packages are stored here:
https://github.com/JuliaLang/julia/tree/master/test/perf/micro
Shall I contact Julia through Twitter, and ask whether they would add
our script to the benchmark list?
Artur
2:49 p.m.
Am 26.02.2018 um 11:40 schrieb Artur T.:
Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
> On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
>
>> Thanks, Artur; btw, there was a bug in the Mandelbrot part. I'll try
>> to get some more work done asap.
>
> This should be it.
That was quick.
Thanks, Jack. I am just trying to find out who the responsible person
for the Julia project is. Any idea?
Could you guys maybe post some comparisons at least between R or Octave
and gretl before going public? I mean, it's your choice and I don't want
to curtail your enthusiasm, but I think it would be interesting.
Shall I contact Julia through Twitter, and ask whether they would add
our script to the benchmark list?
Well I don't know who is "responsible" if there is such a thing, but we
know somebody who's quite involved with the project and could probably
make the connection.
thanks,
sven
3:11 p.m.
On Mon, 26 Feb 2018, Sven Schreiber wrote:
> Thanks, Jack. I am just trying to find out who the responsible
person for
> the Julia project is. Any idea?
Could you guys maybe post some comparisons at least between R or Octave and
gretl before going public? I mean, it's your choice and I don't want to
curtail your enthusiasm, but I think it would be interesting.
Well, I've played a little with those. Long story short: we shred Octave
to bits. Conversely, the R script you find on the Julia website is much,
much faster than ours, since it uses the "compiler" package, which gives R
JIT compilation.
If we could have a LLVM front-end for Hansl, it'd be fantastic.
Unfortunately, I believe we simply don't have the resources to undertake
that project.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
3:22 p.m.
Am 26.02.2018 um 16:11 schrieb Riccardo (Jack) Lucchetti:
On Mon, 26 Feb 2018, Sven Schreiber wrote:
>> Thanks, Jack. I am just trying to find out who the responsible person
>> for the Julia project is. Any idea?
>
> Could you guys maybe post some comparisons at least between R or
> Octave and gretl before going public? I mean, it's your choice and I
> don't want to curtail your enthusiasm, but I think it would be
> interesting.
Well, I've played a little with those. Long story short: we shred Octave
to bits. Conversely, the R script you find on the Julia website is much,
much faster than ours, since it uses the "compiler" package, which gives
R JIT compilation.
Maybe, a the flag "using no JIT" should be added for gretl/hansl then.
These are results I've obtained on my Ubuntu (17.10) machine:
matlab(2014a) R (3.4.3) gretl (2017e)
recursion_fibonacci 53.87 5.00 107.87
parse_integers 13.45 2.00 3.22
userfunc_mandelbrot 4.31 11.00 144.89
recursion_quicksort 8.62 12.00 264.26
iteration_pi_sum 29.68 221.00 1750.23
matrix_statistics 28.27 71.00 17.20
matrix_multiply 26.31 67.00 22.33
print_to_file 711.43 648.00 382.34
I don't have Julia currently installed here...
@Sven: I guess you mean Andreas Noack, right? Could you contact him
about this?
Best,
Artur
6:54 p.m.
Am 26.02.2018 um 16:22 schrieb Artur Tarassow:
Maybe, a the flag "using no JIT" should be added for
gretl/hansl then.
IMHO this would only make sense if there were a switch where a compiler
could be switched on instead (which is not the case of course).
These are results I've obtained on my Ubuntu (17.10) machine:
matlab(2014a) R (3.4.3) gretl (2017e)
recursion_fibonacci 53.87 5.00 107.87
parse_integers 13.45 2.00 3.22
userfunc_mandelbrot 4.31 11.00 144.89
recursion_quicksort 8.62 12.00 264.26
iteration_pi_sum 29.68 221.00 1750.23
matrix_statistics 28.27 71.00 17.20
matrix_multiply 26.31 67.00 22.33
print_to_file 711.43 648.00 382.34
Thanks, Artur, interesting. To me this looks quite OK from gretl's point
of view. The tasks or algorithms where gretl is much slower are those
that aren't really typical or relevant for econometrics and statistics
I'd say. (Anybody feel free to correct me!) Of course that's always the
problem with micro benchmarks in the first place.
Perhaps the only thing where I'd like to know more is the
"recursion_quicksort" kind of thing. (Haven't really looked at the code
yet.) In particular, is there some built-in gretl function that would
normally be used for this kind of task instead of writing a home-cooked
hansl function?
Again, that's not what this micro benchmark thing is about, but in order
to learn about gretl's strenghts and weaknesses it would be useful.
thanks,
sven
7:09 p.m.
On Mon, 26 Feb 2018, Sven Schreiber wrote:
Perhaps the only thing where I'd like to know more is the
"recursion_quicksort" kind of thing. (Haven't really looked at the code
yet.)
In particular, is there some built-in gretl function that would normally be
used for this kind of task instead of writing a home-cooked hansl function?
Yes. The quicksort algorithm is precisely what we use internally for all
sorting, such as for example msortby().
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:37 p.m.
Am 26.02.2018 um 20:09 schrieb Riccardo (Jack) Lucchetti:
On Mon, 26 Feb 2018, Sven Schreiber wrote:
> Perhaps the only thing where I'd like to know more is the
> "recursion_quicksort" kind of thing. (Haven't really looked at the
> code yet.) In particular, is there some built-in gretl function that
> would normally be used for this kind of task instead of writing a
> home-cooked hansl function?
Yes. The quicksort algorithm is precisely what we use internally for all
sorting, such as for example msortby().
Aha, that's what I suspected (hoped). That's the problem with micro
benchmarks: Of course it's also good to know how hansl should _not_ be
used, but the slowness in this benchmark then doesn't tell you anything
about how useful hansl is for a real-world sorting problem - because you
would simply call sort/dsort/msortby and be done, and much faster.
thanks,
sven
6:39 p.m.
Am 26.02.2018 um 16:11 schrieb Riccardo (Jack) Lucchetti:
On Mon, 26 Feb 2018, Sven Schreiber wrote:
If we could have a LLVM front-end for Hansl, it'd be fantastic.
Unfortunately, I believe we simply don't have the resources to undertake
that project.
Yes and no, IMO. Indeed I don't think it makes much sense to reinvent
the wheel and have yet another llvm frontend. But I think the similarity
of julia with respect to hansl and the fact that it's a pretty
self-contained JITting package could be leveraged.
I have started work on a proof of concept that would take the code of a
number-crunching function written in pure (and maybe somewhat
simplified) hansl, would translate it to julia automagically and would
thus use julia as a JITting backend. Of course there's an overhead, but
IMO it's negligible for any significant computation time.
I also have a cool working title for this toy project, but I won't tell
(yet) ;-)
Everybody is welcome to join in, of course.
cheers,
sven
7:08 p.m.
On Mon, 26 Feb 2018, Sven Schreiber wrote:
Am 26.02.2018 um 16:11 schrieb Riccardo (Jack) Lucchetti:
> On Mon, 26 Feb 2018, Sven Schreiber wrote:
> If we could have a LLVM front-end for Hansl, it'd be fantastic.
> Unfortunately, I believe we simply don't have the resources to undertake
> that project.
Yes and no, IMO. Indeed I don't think it makes much sense to reinvent the
wheel and have yet another llvm frontend. But I think the similarity of julia
with respect to hansl and the fact that it's a pretty self-contained JITting
package could be leveraged.
I have started work on a proof of concept that would take the code of a
number-crunching function written in pure (and maybe somewhat simplified)
hansl, would translate it to julia automagically and would thus use julia as
a JITting backend. Of course there's an overhead, but IMO it's negligible for
any significant computation time.
I also have a cool working title for this toy project, but I won't tell (yet)
;-)
Everybody is welcome to join in, of course.
This sounds like a lot of fun, but I have the feeling that in order to run
Hansl code on LLVM going through Julia may not be the best idea ('cause
that's what you'd be doing, since Julia runs on top of LLVM). I suspect
that producing a Hansl->Julia translator is going to be VERY tough, if we
want it to work in more than 90% of arbitrary hansl scripts that you have
out there in the wild.
Instead, we already have a perfecly good lexer+parser combo (genlex.c and
geneval.c in lib/src, if anyone's interested) for assignment and function
evaluation, plus an eccellent tokenizer. As far as my (limited)
understanding of the matter goes, writing a frontend would mean modifiying
those in order to produce IM (LLVM's "machine code"), which you then feed
into the virtual machine.
That said, I am fairly certain that the only person qualified to make a
credible guess on how hard that would be to do is Allin. My guess is that
it'd require several months of very-high-skilled work.
If anyone on the list had some money to spare for a 12-month post-doc
position for a CompSci guy, that'd be just perfect.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
7:55 p.m.
Am 26.02.2018 um 20:08 schrieb Riccardo (Jack) Lucchetti:
On Mon, 26 Feb 2018, Sven Schreiber wrote:
> I have started work on a proof of concept that would take the
code of
> a number-crunching function written in pure (and maybe somewhat
> simplified) hansl, would translate it to julia automagically and would
> thus use julia as a JITting backend.
This sounds like a lot of fun, but I have the feeling that in order
to
run Hansl code on LLVM going through Julia may not be the best idea
('cause that's what you'd be doing, since Julia runs on top of LLVM). I
suspect that producing a Hansl->Julia translator is going to be VERY
tough, if we want it to work in more than 90% of arbitrary hansl scripts
that you have out there in the wild.
I agree with the latter point, and that's exactly why this is _not_ the
aim. I'm not talking about entire scripts, and not talking about
"arbitrary", either. Instead, the aim would be to:
- write a prototype hansl program for the problem at hand
- identify the bottleneck computation
- rewrite this bottleneck in a simple loop-oriented hansl function
(which would be very inefficient in normal hansl usage, but very simple
to write, and very good input for the compiler)
- have the yet-to-be-written tool translate only this one "loopy" hansl
function to julia and execute it there
- leave all the rest in hansl
I'm not saying this approach would cover 100% of all needs, but I would
bet up to 95%. So far I found the automatic translation of matrix- and
math-oriented code from (simplified) hansl to julia quite doable, with a
lot of help from the regsub() function.
It wouldn't be fully automatic because you would have to come up with an
additional specialized hansl function. But from a user's point of view
you would never leave the gretl+hansl world.
credible guess on how hard that would be to do is Allin. My guess is
that it'd require several months of very-high-skilled work.
["it" being: "writing a frontend ... to produce IM (LLVM's
"machine code")]
Yes, I can easily believe that estimate. That's why I think my
simplistic workaround is a viable alternative, because it's so much less
sophisticated and would require perhaps weeks of moderately-high-skilled
work.
cheers,
sven
9:29 a.m.
Am 26.02.2018 um 15:49 schrieb Sven Schreiber:
Am 26.02.2018 um 11:40 schrieb Artur T.:
> Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
>> On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
>>
> Shall I contact Julia through Twitter, and ask whether they would add
> our script to the benchmark list?
Well I don't know who is "responsible" if there is such
a thing, but we
know somebody who's quite involved with the project and could probably
make the connection.
More on the "going public" thing: How shall we proceed? Assuming that
our "perf" code works fine, let's contact those Julia guys to add gretl
on the benchmark list. I guess we have 2 options here: (1) Sven speaks
to A. Noack, and (2) We contact Julia through Twitter.
Best,
Artur
10:25 a.m.
On Tue, 27 Feb 2018, Artur Tarassow wrote:
Am 26.02.2018 um 15:49 schrieb Sven Schreiber:
> Am 26.02.2018 um 11:40 schrieb Artur T.:
>> Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
>>> On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
>>>
>> Shall I contact Julia through Twitter, and ask whether they would add our
>> script to the benchmark list?
> Well I don't know who is "responsible" if there is such a thing, but we
> know somebody who's quite involved with the project and could probably make
> the connection.
More on the "going public" thing: How shall we proceed? Assuming that our
"perf" code works fine, let's contact those Julia guys to add gretl on the
benchmark list. I guess we have 2 options here: (1) Sven speaks to A. Noack,
and (2) We contact Julia through Twitter.
I would think we could do both.
-------------------------------------------------------
Riccardo (Jack) Lucchetti
Dipartimento di Scienze Economiche e Sociali (DiSES)
Università Politecnica delle Marche
(formerly known as Università di Ancona)
r.lucchetti(a)univpm.it
http://www2.econ.univpm.it/servizi/hpp/lucchetti
-------------------------------------------------------
1:47 p.m.
Am 27.02.2018 um 11:25 schrieb Riccardo (Jack) Lucchetti:
On Tue, 27 Feb 2018, Artur Tarassow wrote:
> Am 26.02.2018 um 15:49 schrieb Sven Schreiber:
>> Am 26.02.2018 um 11:40 schrieb Artur T.:
>>> Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
>>>> On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
>>>>
>>> Shall I contact Julia through Twitter, and ask whether they would
>>> add our script to the benchmark list?
>
>> Well I don't know who is "responsible" if there is such a thing,
but
>> we know somebody who's quite involved with the project and could
>> probably make the connection.
>
> More on the "going public" thing: How shall we proceed? Assuming that
> our "perf" code works fine, let's contact those Julia guys to add
> gretl on the benchmark list. I guess we have 2 options here: (1) Sven
> speaks to A. Noack, and (2) We contact Julia through Twitter.
I would think we could do both.
Ok, I've send a request to Julia via Twitter.
Artur
8:17 a.m.
Ok, Julia has replied through Twitter. They sent us this link:
https://github.com/JuliaLang/julia/blob/master/CONTRIBUTING.md
The micro-benchmark files are stored here:
https://github.com/JuliaLang/julia/tree/master/test/perf/micro
Since I've no idea about git, could anyone please do the necessary
steps? Thank you.
The "perf.inp" file is attached.
Best,
Artur
Am 27.02.2018 um 11:25 schrieb Riccardo (Jack) Lucchetti:
On Tue, 27 Feb 2018, Artur Tarassow wrote:
> Am 26.02.2018 um 15:49 schrieb Sven Schreiber:
>> Am 26.02.2018 um 11:40 schrieb Artur T.:
>>> Am 25.02.2018 um 20:25 schrieb Riccardo (Jack) Lucchetti:
>>>> On Sun, 25 Feb 2018, Riccardo (Jack) Lucchetti wrote:
>>>>
>>> Shall I contact Julia through Twitter, and ask whether they would
>>> add our script to the benchmark list?
>
>> Well I don't know who is "responsible" if there is such a thing,
but
>> we know somebody who's quite involved with the project and could
>> probably make the connection.
>
> More on the "going public" thing: How shall we proceed? Assuming that
> our "perf" code works fine, let's contact those Julia guys to add
> gretl on the benchmark list. I guess we have 2 options here: (1) Sven
> speaks to A. Noack, and (2) We contact Julia through Twitter.
I would think we could do both.
2242
days inactive
2245
days old
19 comments
4 participants
participants (4)
-
Artur T. -
Artur Tarassow
-
Riccardo (Jack) Lucchetti -
Sven Schreiber