On 14.08.2019 05:44, Marcin Błażejowski wrote:
On 13.08.2019 21:28, Allin Cottrell wrote:
> On Tue, 13 Aug 2019, Marcin Błażejowski wrote:
>
>> On 13.08.2019 19:22, Allin Cottrell wrote:
>>> On Tue, 13 Aug 2019, Marcin Błażejowski wrote:
>>>
>>>> It looks like the problem with concurent access to the file. I solved
>>>> the issue, but the solution is quite numb:
>>>> ##########################
>>>> loop i=0..$mpisize-1 --quiet
>>>> if ($mpirank == i)
>>>> MODEL = bread("MODEL")
>>>> else
>>>> sleep(1)
>>>> endif
>>>> endloop
>>>> ##########################
>>> Possibly relevant questions: Is the bundle file gzipped or just plain
>>> XML? How/when is the bundle file created?
>> Allin,
>>
>> this is a plain XML created by bwrite() few lines before bread() is
>> used:
> OK. I assume the bundle is being written by one process, to be read by
> all, or all the others?
>
> If so, you probably need to add an MPI barrier, otherwise some
> processes may try to start reading the file before the writer has
> finished writing it.
>
> We don't have a simple MPI barrier as such in our hansl implementation
> of MPI (maybe we should have such), but you could
> just broadcast a scalar from the process that's writing the file.
> Something like:
>
> <hansl>
> # write bundle HERE
> scalar bundle_ready
> if $mpirank == 0 # or whoever is writing
> bundle_ready = 1
> endif
> mpibcast(&bundle_ready)
> if bundle_ready
> # read bundle HERE
> endif
> <hansl>
Ok, thanks Allin. In my case "# write bundle HERE" is done before MPI
block, but I understand it doesn't matter?
Hmm, the workaround doesn't work.
BTW: can I suppress messages from MPI nodes about loading dataset when
MPI '--send-data' flag is used?
Marcin
--
Marcin Błażejowski