Skip to content

Topological capacity always zero when running Legion on ogbn-papers100M #7

Description

@Yash685

Thank you for the great work on the Legion paper and open-sourcing the codebase. I’m currently trying to apply it on ogbn-papers100M , particularly to evaluate the time taken by the cache cost model, but I’ve encountered an issue where the topology capacity is always reported as zero, and the topology transactions show up as NaN. The steps that I followed were:

  1. Converted OGBN dataset to edge list:
from ogb.nodeproppred import NodePropPredDataset
import numpy as np
dataset = NodePropPredDataset(name="ogbn-papers100M", root="/data/")
graph, _ = dataset[0]
edge_index = graph['edge_index']
if hasattr(edge_index, 'numpy'):
    edge_list = edge_index.numpy().T
else:
    edge_list = edge_index.T
np.savetxt("edge_list.txt", edge_list, fmt="%d") 
  1. Generate partitions and training/validation/test sets:
 ./gen_legion_xtrapulp_format paper100m edge_list.txt
python gen_sets.py --dataset_name paper100m 
  1. Run the sampling server:
python legion_server.py \
  --dataset_path dataset \
  --dataset_name paper100m \
  --train_batch_size 8000 \
  --fanout [15,10,5] \
  --gpu_number 1 \
  --epoch 2 \
  --cache_memory 800000

Observations:

  • Topo capacity/split is printed as 0.
  • All values of h_edge_index in the topology init kernel are zero.
  • Resulting topology transaction stats are NaN

Is there something wrong with the preprocessing or check I might have missed?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions