At SACLA, we have only one fat-memory node (1TB RAM) and a time limit of 24 hr for a job. Thus, the memory and time requirement of post-refinement limits the size of the dataset we can process.
I have a suggestion to make post-refinement more scalable on multiple nodes.
- refine images in parallel on multiple worker nodes using a common reference
- write the refined parameters to the (shared) disk
- the master node waits for all worker nodes
- the master node merges all images using refined partialities and creates a new reference
- distribute the new merged reference to worker nodes and repeat from 1.
This can be ideally implemented using MPI, but more easily by shell scripts using GNU parallel with minimum modifications to cppxfel.
At SACLA, we have only one fat-memory node (1TB RAM) and a time limit of 24 hr for a job. Thus, the memory and time requirement of post-refinement limits the size of the dataset we can process.
I have a suggestion to make post-refinement more scalable on multiple nodes.
This can be ideally implemented using MPI, but more easily by shell scripts using GNU parallel with minimum modifications to cppxfel.