feat(kperf): support exact count and configurable batch size - #1312
Open
xinWeiWei24 wants to merge 3 commits into
Open
feat(kperf): support exact count and configurable batch size#1312xinWeiWei24 wants to merge 3 commits into
xinWeiWei24 wants to merge 3 commits into
Conversation
Drop the -first/-retry name suffix so each batch uses a stable set name "<name>-b<batch>", making object names predictable for kperf load profiles to target by prefix. Replace the rename-based retry (which left partial objects from the failed attempt as orphans) with a bounded retry loop that deletes the partial batch before re-creating, so each batch ends up with exactly COUNT objects and accurate progress accounting.
Allow callers to specify the exact object count and batch size while preserving the existing GiB-based ingestion behavior.
xinWeiWei24
requested review from
Liyu Ma (liyu-ma),
Vittoria (vittoriasalim) and
Lei Yao (wonderyl)
as code owners
September 4, 2026 03:48
|
For reviewers only: reply |
| # Each batch uses the set name "<name>-b<batch>", so object names are predictable and | ||
| # kperf load profiles can target a batch by its prefix. | ||
| RunKperfDataIngest = lambda dataType: str, namespace: str, name: str, totalGiB: str, qps: int = DEFAULT_QPS, burst: int = DEFAULT_BURST, size: int = DEFAULT_SIZE_BYTES, groupSize: int = DEFAULT_GROUP_SIZE -> steps.Step { | ||
| RunKperfDataIngest = lambda dataType: str, namespace: str, name: str, totalGiB: str = "0", qps: int = DEFAULT_QPS, burst: int = DEFAULT_BURST, size: int = DEFAULT_SIZE_BYTES, groupSize: int = DEFAULT_GROUP_SIZE, total: int = 0, batchSize: int = 2000 -> steps.Step { |
Collaborator
There was a problem hiding this comment.
Should disable totalGiB and total are both set?
| # Each batch uses the set name "<name>-b<batch>", so object names are predictable and | ||
| # kperf load profiles can target a batch by its prefix. | ||
| RunKperfDataIngest = lambda dataType: str, namespace: str, name: str, totalGiB: str, qps: int = DEFAULT_QPS, burst: int = DEFAULT_BURST, size: int = DEFAULT_SIZE_BYTES, groupSize: int = DEFAULT_GROUP_SIZE -> steps.Step { | ||
| RunKperfDataIngest = lambda dataType: str, namespace: str, name: str, totalGiB: str = "0", qps: int = DEFAULT_QPS, burst: int = DEFAULT_BURST, size: int = DEFAULT_SIZE_BYTES, groupSize: int = DEFAULT_GROUP_SIZE, total: int = 0, batchSize: int = 2000 -> steps.Step { |
Collaborator
There was a problem hiding this comment.
move total and totalGiB closer, so that reader can easily spot them together.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow callers to specify the exact object count and batch size while preserving the existing GiB-based ingestion behavior.