Thank you for the great work on the Legion paper and open-sourcing the codebase. I’m currently trying to apply it on ogbn-papers100M , particularly to evaluate the time taken by the cache cost model, but I’ve encountered an issue where the topology capacity is always reported as zero, and the topology transactions show up as NaN. The steps that I followed were:
- Converted OGBN dataset to edge list:
from ogb.nodeproppred import NodePropPredDataset
import numpy as np
dataset = NodePropPredDataset(name="ogbn-papers100M", root="/data/")
graph, _ = dataset[0]
edge_index = graph['edge_index']
if hasattr(edge_index, 'numpy'):
edge_list = edge_index.numpy().T
else:
edge_list = edge_index.T
np.savetxt("edge_list.txt", edge_list, fmt="%d")
- Generate partitions and training/validation/test sets:
./gen_legion_xtrapulp_format paper100m edge_list.txt
python gen_sets.py --dataset_name paper100m
- Run the sampling server:
python legion_server.py \
--dataset_path dataset \
--dataset_name paper100m \
--train_batch_size 8000 \
--fanout [15,10,5] \
--gpu_number 1 \
--epoch 2 \
--cache_memory 800000
Observations:
- Topo capacity/split is printed as 0.
- All values of h_edge_index in the topology init kernel are zero.
- Resulting topology transaction stats are NaN
Is there something wrong with the preprocessing or check I might have missed?
Thank you for the great work on the Legion paper and open-sourcing the codebase. I’m currently trying to apply it on ogbn-papers100M , particularly to evaluate the time taken by the cache cost model, but I’ve encountered an issue where the topology capacity is always reported as zero, and the topology transactions show up as NaN. The steps that I followed were:
from ogb.nodeproppred import NodePropPredDataset import numpy as np dataset = NodePropPredDataset(name="ogbn-papers100M", root="/data/") graph, _ = dataset[0] edge_index = graph['edge_index'] if hasattr(edge_index, 'numpy'): edge_list = edge_index.numpy().T else: edge_list = edge_index.T np.savetxt("edge_list.txt", edge_list, fmt="%d")Observations:
Is there something wrong with the preprocessing or check I might have missed?