refactor(ops): route basic Llama through canonical InfiniOps

[voltjia]

What

• Route the NVIDIA basic Llama prefill, decode, RoPE, KV-cache update,
  greedy-sampling, residual RMSNorm, and SwiGLU paths through canonical
  InfiniOps C++ APIs.
• Include the paged-attention decode and prefill changes formerly reviewed in
  refactor(ops): migrate paged attention to canonical InfiniOps #1481. This PR now targets main; refactor(ops): migrate paged attention to canonical InfiniOps #1481 is closed as superseded.
• Remove InfiniLM-suffixed InfiniOps references and legacy InfiniOP C API
  adapters from the validated InfiniCore path.
• Use the generated PyTorch Argmax provider on NVIDIA. The InfiniOps external
  build enables only argmax, InfiniCore selects provider implementation 8,
  and the gitlink remains at 47c1c496, before InfiniOps issue/867 fix cpu malloc #918.
• Keep the public InfiniCore C++ and Python APIs unchanged. Compatibility
  adaptation remains inside InfiniCore providers.

Migration

InfiniCore route	Previous dependency	Canonical InfiniOps route	Provider adaptation
KV-cache insertion	KvCachingInfinilm / legacy InfiniOP adapter	ReshapeAndCacheFlash	Map InfiniCore cache metadata to the vLLM-style cache update contract.
Paged prefill	PagedAttentionPrefillInfinilm / legacy InfiniOP adapter	FlashAttnVarlenFunc	Map the basic Llama varlen layout to the FlashAttention public contract.
Paged decode	PagedAttentionInfinilm fallback	FlashAttnWithKvcache	Use the canonical path for the validated basic Llama cache layout.
RoPE	RotaryEmbeddingInfinilm / legacy InfiniOP adapter	RotaryEmbedding	Preserve the public InfiniCore argument order while calling the vLLM-aligned operator.
Greedy sampling (top_k=1)	RandomSampleInfinilm / legacy InfiniOP adapter	Argmax	Reduce final logits along the vocabulary dimension with the generated PyTorch provider.
Residual RMSNorm	deprecated AddRmsNorm	FusedAddRmsNorm	Copy only for non-aliasing public outputs, then use the canonical in-place vLLM contract. The basic Llama path aliases and adds no extra copies.
SwiGLU	deprecated Swiglu(a, b)	Copy + SiluAndMul	Pack [gate=b, up=a], then evaluate vLLM's silu(gate) * up contract.

Alignment Evidence

The selected targets follow InfiniOps' alignment policy: prefer a public
PyTorch Python API, then a serving-framework wrapper, then ONNX or a
library-owned public interface. InfiniOps applies the accepted C++ ordering
adaptation: tensor inputs, attributes, then writable outputs.

InfiniOps API	Upstream contract	Alignment target
Embedding	embedding(input, weight, padding_idx=None, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, sparse=False)	PyTorch torch.nn.functional.embedding
RmsNorm	rms_norm(out, input, weight, epsilon); InfiniOps moves out to the output group	vLLM _custom_ops.rms_norm
Gemm	Gemm(A, B, C?, alpha, beta, transA, transB) -> Y	ONNX Gemm
RotaryEmbedding	rotary_embedding(positions, query, key, head_size, cos_sin_cache, is_neox, rope_dim_offset=0, inverse=False)	vLLM _custom_ops.rotary_embedding
ReshapeAndCacheFlash	reshape_and_cache_flash(key, value, key_cache, value_cache, slot_mapping, kv_cache_dtype, k_scale, v_scale)	vLLM _custom_ops.reshape_and_cache_flash
FlashAttnVarlenFunc	flash_attn_varlen_func(q, k, v, cu_seqlens_q, cu_seqlens_k, max_seqlen_q, max_seqlen_k, ..., return_attn_probs=False)	FlashAttention public interface
FlashAttnWithKvcache	flash_attn_with_kvcache(q, k_cache, v_cache, k=None, v=None, ..., return_softmax_lse=False)	FlashAttention public interface
FusedAddRmsNorm	fused_add_rms_norm(input, residual, weight, epsilon)	vLLM _custom_ops.fused_add_rms_norm
Copy	Tensor.copy_(src, non_blocking=False); InfiniOps represents the receiver as trailing out	PyTorch Tensor.copy_
SiluAndMul	SiluAndMul.forward(x), where silu(x[..., :d]) * x[..., d:]	vLLM SiluAndMul
Argmax	torch.argmax(input, dim=None, keepdim=False)	PyTorch torch.argmax

Scope

This PR targets the NVIDIA basic Llama architecture flow used by InfiniLM with
paged FlashAttention and greedy sampling. Qwen3 uses this same validated path.
Non-greedy sampling, unsupported attention layouts, other model families, and
other devices remain out of scope.

InfiniOps #918 is not a dependency of this PR. It remains an optional native
NVIDIA Argmax implementation; this PR uses the generated PyTorch provider.

Screenshots: N/A (backend integration only).

Validation

Run on ssh nvidia in accelerator-dev/nvidia:latest on one NVIDIA
A100-SXM4-80GB:

• git diff --check origin/main...HEAD passed.
• clang-format 16.0.6 --dry-run --Werror passed for every modified C++ file.
• ruff 0.15.20 check and ruff 0.15.20 format --check passed for the modified
  Python test.
• The focused InfiniOps/InfiniCore build passed with WITH_TORCH=ON and
  INFINI_OPS_TORCH_OPS=argmax. The built library references
  at::argmax_out; no native NVIDIA Argmax source from issue/867 fix cpu malloc #918 is present.
• Source scans found no InfiniLM-suffixed InfiniOps references in the validated
  InfiniCore path.
• An LD_PRELOAD trap covering 607 legacy infiniop* C API symbols across 154
  descriptor families did not fire during inference.
• TinyLlama-1.1B-Chat inference exited 0 on the same canonical path.
• Qwen3-0.6B inference exited 0 and produced 64 coherent greedy tokens. It
  matched PyTorch for the first 6 generated tokens, then selected " just"
  where PyTorch selected " said". At that first divergence, PyTorch BF16
  logits are exactly tied at 22.875 for both tokens, so exact sequence
  equality is not a stable precision criterion for this prompt.
• Qwen3-4B inference exited 0 and its 64 generated tokens matched PyTorch
  Transformers exactly (64/64), including the decoded text.

Commands used for both Qwen3 models:

python3 examples/test_infer.py \
    --device=nvidia \
    --model=/tmp/standard-llama/Qwen3-0.6B \
    --enable-paged-attn \
    --attn=flash-attn \
    --top-k=1 \
    --top-p=1.0 \
    --temperature=1.0 \
    --max-new-tokens=64 \
    --prompt=Hello

python3 examples/test_infer.py \
    --device=nvidia \
    --model=/tmp/standard-llama/Qwen3-4B \
    --enable-paged-attn \
    --attn=flash-attn \
    --top-k=1 \
    --top-p=1.0 \
    --temperature=1.0 \
    --max-new-tokens=64 \
    --prompt=Hello