• Spike-loss and random crashes experienced with with process numbers that were not a power of two. (OpenMPI 1.4.3)
• Random communication freezes for large messages (i.e. caused by synchronization in the network → many spikes in short time intervals). (OpenMPI 1.5.4)
• Still getting random freezes when running v0.8.0-beta on Ubuntu 14.04 LTS with the stock openmpi (Open MPI 1.6.5)

I traced the freezes to SyncBuffer at high firing rates or high synchrony. In most cases it seems as if some processes do not recover from MPI_Allgather when resending the buffer after an overflow. To reproduce run

mpirun -n 4 ./sim_background --dir /tmp --wee 0.17

or for instance

mpirun -n 4 ./sim_poisson --kappa 20000

Note, that -n 2 seems to work and 3 or 5 cause other weird problems. Looks like a bug in openmpi. When compiled against MPICH these problems do not occur.