Skip to content

Add initial T5Gemma S/S support#947

Draft
maasir554 wants to merge 6 commits into
google:devfrom
maasir554:t5gemma-support
Draft

Add initial T5Gemma S/S support#947
maasir554 wants to merge 6 commits into
google:devfrom
maasir554:t5gemma-support

Conversation

@maasir554

@maasir554 maasir554 commented Jul 8, 2026

Copy link
Copy Markdown

Summary

Adds initial T5Gemma S/S support to gemma.cpp.

This includes:

  • T5Gemma S/S model config and tensor metadata
  • HF safetensors -> SBS conversion through python/convert_from_safetensors.py
  • Encoder-decoder runtime path for fresh seq2seq generation
  • BF16 default conversion path for correctness/parity
  • Basic docs and focused config/tensor/cache tests

Scope

This PR targets T5Gemma v1 S/S only.

Out of scope:

  • T5Gemma 2
  • Other T5Gemma sizes
  • Multiturn/cache reuse with changing encoder inputs
  • Fully optimized T5-specific attention/logits kernels

Local Validation

  • bazel test -c opt //:configs_test //:tensor_info_test //:kv_cache_test
  • Local optimized smoke test with t5gemma-s-s-it
    • prompt: Hello
    • output begins: I am a new user, ...
    • local speed: roughly 30-35 generated tok/s on MacBook M1, 8GB RAM

Notes

Opened as draft to get early maintainer feedback on scope, naming, and whether this should be split further.

Additional Note: //:gemma_lib_test currently fails locally for me in an existing GemmaAttentionFlash/EMU128 golden mismatch on Apple Silicon; this reproduced outside the T5Gemma changes.

Related to #661

@google-cla

google-cla Bot commented Jul 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@maasir554 maasir554 marked this pull request as draft July 8, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant