On Fri, 13 Jan 2017, Sven Schreiber wrote:
 Am 13.01.2017 um 20:48 schrieb Allin Cottrell:
> On Fri, 13 Jan 2017, Sven Schreiber wrote:
>> It's great to be able to pass bundles around. However, I still don't
>> get how the bundles (array) can be injected into the mpi block, in
>> your example you're constructing them on the fly within the block
>> (your whole script being the block here, obviously).
> 
> Why do you need them to be injected? That is, why not just do everything
> "inside" MPI?
 Because I want to encapsulate the stuff in functions, to do something like 
 this:
 <hansl-future>
 include SVAR.gfn
 include gretlparallel.gfn # doesn't exist yet
 bundles bb = defarray( setupSVAR(...longrunrestr...), \
 setupSVAR(...shortrunrest...) )
 string content = "SVAR_estimate(&mybundle)"
 content ~= sprintf("\nSVAR_boot(&mybundle)\n")
 bundles bbout = gretlparallel(content, bb, "mybundle")
 </hansl-future> 
Thanks, I now have a clearer picture of what you're aiming to do, 
and it's quite interesting.
I'm attaching a new variant of my own. I've cut back on printing, 
error checking and options to try to expose the overall structure as 
clearly as possible. Besides that there are two main points that you 
might like to consider.
1) MPI stuff: you don't need to write out N temporary script files, 
one will do nicely. The only rank-specific point in the script is 
the name of the bundle to read/write and that can be constructed 
conditional on $mpirank. Also, echo and messages are off by default 
in an MPI block; there's no need to turn them off explicitly.
2) Per-process output files: I think that for most users most of the 
time it would be more user-friendly to gather the content of these 
files and consolidate the output, so that's what I've done. I guess 
one could have an option switch in this regard.
 (BTW, I guess the MPI setup also supports accessing the same 
 libraries on the system concurrently from the different 
 workers/threads, including the R lib through 'foreign'?) 
Should do, yes.
> 2) I think you may find it's much nicer if you can put the
common
> working code block into the MPI section. That way you just have to write
> hansl, not squirt it though printf as in
 Yes, but: First it's already done, and not more needed. 
Point taken, in regard to the "boilerplate" code inside your 
function. But what about the user code? That has to be supplied as a 
string, so the user must write hansl-generating hansl. Easy enough 
in a trivial case but gets nasty if you're using printf or sprintf.
I haven't done anything about this in my current variant, but if it 
were my package I'd probably want to give the option of supplying 
the user-code via the name of a script file rather than a hansl 
string.
Last small point: my variant cleans up its temporary files, if we 
don't want to preserve tham for debugging purposes.
Allin