A GPipe implementation in PyTorch
-
Updated
Jul 25, 2024 - Python
#
gpipe
Here are 8 public repositories matching this topic...
Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
-
Updated
May 24, 2022 - Python
Development of Project HPGO | Hybrid Parallelism Global Orchestration
rust machine-learning tensorflow pytorch data-parallelism model-parallelism distributed-training pipedream gpipe pipeline-parallelism
-
Updated
Mar 26, 2021
Simulated Multi-GPU inference engine implementing Megatron-style Tensor Parallelism, GPipe Pipeline Parallelism, and KV Cache Sharding from scratch. Features bit-identical fidelity validation on Qwen2-0.5B weights and analytical communication cost modeling.
distributed-systems hpc distributed-computing pytorch scaling multi-gpu inference-optimization gpipe pipeline-parallelism kv-cache megatron-lm tensor-parallelism llm qwen2 torch-distributed
-
Updated
Jul 29, 2026 - Python
A fully distributed hyperparameter optimization tool for PyTorch DNNs
-
Updated
Jan 12, 2022 - Python
Improve this page
Add a description, image, and links to the gpipe topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the gpipe topic, visit your repo's landing page and select "manage topics."