Registered KMU fanout can expose a false-idle cycle before the final CTA reaches a core

[ForeverHYX]

Summary

In a multi-output configuration, the final CTA launch can remain inside a
registered VX_kmu_arb output buffer after VX_kmu clears running. The
device-wide busy aggregation currently observes the KMU producer and the
registered downstream core state, but not the request while it is owned by the
fanout buffer. This can expose a false-idle cycle before the destination core
accepts the CTA.

sim/rtlsim/processor.cpp returns from its launch wait on the first low sample
of device_->busy, so that single-cycle gap can be reported to software as
kernel completion.

Affected revision and environment

• Vortex master: d76b7f24e658867ab57e3942d7c648c3e6af072d

• VORTEX_VERSION=3.0, TOOLCHAIN_REV=v3.0

• Ubuntu 24.04.4 LTS, x86-64

• Verilator 5.046 (v5.046-55-g1264184fb)

• LLVM 20.1.8 from vortexgpgpu/llvm revision
  4c836512a60c0b965bb298104e084cd0f07ca962

• Toolchain installed under $HOME/tools with the project toolchain
  installer

• Out-of-tree configuration:

  ../configure --xlen=32 --tooldir=$HOME/tools

Minimal RTL reproducer

Instantiate VX_kmu_arb with one input, two outputs, and OUT_BUF=3:

1. Reset the arbiter.
2. Drive valid_in=1 and make both destinations ready for one rising edge, so
   the registered fanout accepts one request.
3. Deassert valid_in and backpressure both destinations before the next
   transfer.
4. Hold the output blocked for several cycles.

At step 3 the observed ownership state is:

input valid = 0
output valid = 0x1
destination has not accepted the request

The request is real and remains stable in the fanout, but the upstream KMU can
already have busy=0 and registered child busy has not yet propagated upward.
There is no arbiter occupancy signal for the parent to include in its busy
chain.

A Verilator unit test for this exact handoff is ready and will be submitted
with the fix. It runs from an out-of-tree build with:

CCACHE_DISABLE=1 make -C hw/unittest/kmu_arb run

As a sensitivity check, modeling the existing input-only coverage produces
this deterministic failure:

buffered handoff: in_valid=0 out_valid=0x1 pending=0
FAIL: pending must bridge the source-to-buffer ownership transfer

With output-buffer occupancy included, the same state is:

buffered handoff: in_valid=0 out_valid=0x1 pending=1
PASSED: KMU fanout pending covers input, buffering, and output

Expected behavior

Device busy must remain continuously asserted from launch until every CTA has
left all KMU fanout buffers and downstream execution has taken ownership. A
registered handoff must not create an observable idle cycle.

Actual behavior and impact

The device can report idle while the final CTA is still buffered. Depending on
the host and workload timing, this can cause:

• premature kernel return;
• stale, partially updated, or uninitialized output data;
• result mismatches, including NaNs when later arithmetic consumes missing
  intermediate values;
• a following launch or configuration sequence overlapping the tail of the
  previous launch;
• instruction and cycle counts that are too small because measurement stops
  before execution drains.

This is timing- and topology-dependent. A single-core configuration generally
does not enable the registered multi-output fanout, and other active busy
sources can sometimes mask the gap. It is not a timeout problem: extending a
watchdog does not help when the host has already accepted a false idle as
completion.

This is related in area but distinct from #366: that report observes a
VCK5000 CP command that never retires, while this reproducer observes a launch
that has asserted busy and is then reported complete before its buffered final
CTA reaches a core.

Proposed fix

Expose a pending output from VX_kmu_arb that covers both input and output
valid state, and include it in the Vortex, cluster, socket, and applicable core
busy aggregations. The accompanying unit test backpressures a registered output
for multiple cycles and verifies continuous pending coverage through input,
buffered, transfer, and drained states.