NonZero Operator - #52
Conversation
sanjibansg
left a comment
sanjibansg
left a comment
There was a problem hiding this comment.
Thanks for the implementation, couple of comments:
| out << SP << "auto nonZeroCountDev_" << fNY << " = alpaka::allocBuf<std::size_t, Idx>(devAcc, Ext1D::all(Idx{1}));\n"; | ||
| out << SP << "auto nonZeroCountHost_" << fNY << " = alpaka::allocBuf<std::size_t, Idx>(hostAcc, Ext1D::all(Idx{1}));\n"; | ||
| out << SP << "auto const workDivNonZero_" << fNY << " = sofie_workdiv(Vec::all(Idx{1}));\n"; | ||
| out << SP << "alpaka::exec<Acc>(queue, workDivNonZero_" << fNY << ", nonZeroKernel_" << fNY << ", alpaka::getPtrNative(deviceBuf_" << fNX << "), alpaka::getPtrNative(bufDev_" << fNY << "), alpaka::getPtrNative(nonZeroCountDev_" << fNY << "));\n"; |
There was a problem hiding this comment.
can we have create task and then enqueue paradigm like we have for the other operators? alpaka::wait is expensive operation, so doing that after every operator will slow down the process.
There was a problem hiding this comment.
Changed the kernel launch to createTaskKernel + enqueue. The wait is still required here because the NonZero count is copied back to the host and immediately used to determine the dynamic output shape, avoiding it would require keeping the count and shape handling on the GPU or redesigning the dynamic-output flow so the host does not need the count immediately
| op += SP + SP + SP + "auto const i = alpaka::getIdx<alpaka::Grid, alpaka::Threads>(acc)[0];\n"; | ||
| op += SP + SP + SP + "if (i != 0u) return;\n"; | ||
| op += SP + SP + SP + "std::size_t n = 0;\n"; | ||
| op += SP + SP + SP + "for (std::size_t j = 0; j < " + std::to_string(fInputLength) + "u; ++j) if (input[j] != static_cast<T>(0)) ++n;\n"; |
There was a problem hiding this comment.
using if in GPU code could be an issue, can there be any other algorithmic way to do this?
There was a problem hiding this comment.
Removed the branch from the counting pass by accumulating the boolean condition directly. A branch remains in the output-writing pass, removing that would require a different prefix-scan algorithm.
2 participants
Adds Alpaka support for ONNX NonZero with parser registration and CUDA tests. Also updates RModel_ALPAKA dynamic-tensor handling so buffers use the correct type, runtime-sized tensors are not allocated before their shape is known, and dynamic model outputs use bufDev_ instead of deviceBuf_. All Alpaka tests pass.