DynaMPI

CI Status

---

Header-only C++20 library for efficient manager–worker dynamic load distribution over MPI.

Licensed under the Apache License 2.0.

Usage

Static number of tasks

#include <cassert>
#include <dynampi/dynampi.hpp>

auto worker_task = [](size_t task) -> size_t { return task * task; };
auto result = dynampi::mpi_manager_worker_distribution<size_t>(4, worker_task);
if (result.has_value()) {
  // Manager: one result per task. Default distributor is hierarchical and
  // unordered — use NaiveMPIWorkDistributor or LockFreeMPIWorkDistributor
  // if you need results in task-index order.
  assert(result->size() == 4);
}

The default distributor is MPIDynamicWorkDistributor (hierarchical). Pass another distributor as a template argument if needed (e.g. NaiveMPIWorkDistributor for strictly ordered results on small communicator sizes).

Dynamic / incremental tasks

When the task set is not known up front, use MPIDynamicWorkDistributor and alternate insert_tasks with run_tasks / finish_remaining_tasks:

#include <dynampi/dynampi.hpp>

using Task = int;
using Result = std::vector<int>;

auto worker_task = [](Task task) -> Result {
  return Result{task, task * task, task * task * task};
};

dynampi::MPIDynamicWorkDistributor<Task, Result> distributor(worker_task);
if (distributor.is_root_manager()) {
  distributor.insert_tasks({1, 2, 3, 4, 5});
  auto results = distributor.finish_remaining_tasks();
  // results.size() == 5

  distributor.insert_tasks({6, 7, 8});
  results = distributor.finish_remaining_tasks();
  // results.size() == 3
}

With auto_run_workers = true (the default), non-manager ranks enter the worker loop in the constructor. Optional compile-time features include task prioritization (naive distributor), statistics tracking, and custom MPI datatypes via dynampi::MPI_Type.

See the documentation for distributor choice (naive, hierarchical, lock-free RMA) and configuration.

Installation

DynaMPI depends only on MPI. Copy include/ into your project, or consume it with CMake:

include(FetchContent)
FetchContent_Declare(
    dynampi
    GIT_REPOSITORY https://github.com/ryanstocks00/DynaMPI.git
    GIT_TAG        main
)
FetchContent_MakeAvailable(dynampi)
target_link_libraries(my_target PRIVATE dynampi)