Skip to content

[0029][0031]In linalg Mul/MulAdd functions, the memory backing the Matrix and Bias buffers must be read-only - #507

Open
Anupama Chandrasekhar (anupamachandra) wants to merge 1 commit into
microsoft:mainfrom
anupamachandra:anupamac/clarify-matrix-bias-BAB
Open

[0029][0031]In linalg Mul/MulAdd functions, the memory backing the Matrix and Bias buffers must be read-only#507
Anupama Chandrasekhar (anupamachandra) wants to merge 1 commit into
microsoft:mainfrom
anupamachandra:anupamac/clarify-matrix-bias-BAB

Conversation

@anupamachandra

@anupamachandra Anupama Chandrasekhar (anupamachandra) commented May 29, 2025

Copy link
Copy Markdown
Contributor

@anupamachandra Anupama Chandrasekhar (anupamachandra) changed the title [0029][0031]In Mul/MulAdd the memor backing the Matrix adn Bias buffers must be read-only [0029][0031]In linalg Mul/MulAdd functions, the memory backing the Matrix and Bias buffers must be read-only May 29, 2025

@pow2clk Gregory Roth (pow2clk) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have anything against this apart from the minor quibble about example code comments.

When I took an initial run at this, I had significantly more changes to the API itself as a result of disallowing RWMatrixRefs which results in much better error messsages when RW types are used: pow2clk@ea67c02 Sorry I didn't link this to you sooner.

matrix is stored in memory, while the vector comes from a variable. The Mul
function expects the memory backing the matrix object to be read-only
(for efficient loads and hardware optimizations) and, therefore, the matrix
must be of MatrixRef type (and not RWMatrixRef type).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expected this to result in changing the interface of these functions so they take MatrixRef and not MatrixRefImpl.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

bias vector (loaded from memory) to the result.
bias vector (loaded from memory) to the result. Similar to the the matrix
operand, the memory backing the bias vector must be read-only, and therefore an
object of `VectorRef` type (and not `RWVectorRef` type).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above except without the ability to specify RWVectorRef here, the type is not needed at all.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

ByteAddressBuffer inputMatrix0; // note read-only buffer
ByteAddressBuffer inputMatrix1; // note read-only buffer
ByteAddressBuffer biasVector0; // note read-only buffer
ByteAddressBuffer biasVector1; // note read-only buffer

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think one comment at the top of the block pointing out that they are all read-only buffers is sufficient.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly since ByteAddressBuffer is already a read only buffer I don't think any of these comments are needed. Its belaboring the point

matrix is stored in memory, while the vector comes from a variable. The Mul
function expects the memory backing the matrix object to be read-only
(for efficient loads and hardware optimizations) and, therefore, the matrix
must be of MatrixRef type (and not RWMatrixRef type).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

bias vector (loaded from memory) to the result.
bias vector (loaded from memory) to the result. Similar to the the matrix
operand, the memory backing the bias vector must be read-only, and therefore an
object of `VectorRef` type (and not `RWVectorRef` type).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

5 participants