Skip to content

multiprocessing hangs in iterative solver #24

Description

@bonfus

For some reason, the second time (?!) the code hits this line:

https://github.com/stefanch/sGDML/blob/master/sgdml/utils/desc.py#L345

during sgdml all, multiprocessing imap blocks the execution and ctrl+c gives this backtrace:

^C[CRIT] Traceback (most recent call last):
         File "/usr/lib/python3.9/multiprocessing/pool.py", line 853, in next
           item = self._items.popleft()
       IndexError: pop from an empty deque

       During handling of the above exception, another exception occurred:

       Traceback (most recent call last):
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/cli.py", line 1083, in train
           model = gdml_train.train(
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/train.py", line 927, in train
           R_desc, R_d_desc = desc.from_R(
         File "/home/pie/Venvs/sgdmlenv/lib/python3.9/site-packages/sgdml/utils/desc.py", line 345, in from_R
           map_func(partial(_from_r, lat_and_inv=lat_and_inv), R)
         File "/usr/lib/python3.9/multiprocessing/pool.py", line 858, in next
           self._cond.wait(timeout)
         File "/usr/lib/python3.9/threading.py", line 312, in wait
           waiter.acquire()
       KeyboardInterrupt

Disabling parallelism in this function is an acceptable workaround (not much time spent here).

Using numpy compiled with MKL support and 12 cores.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions