Skip to content

test(eval): record eval run at 1ad1ed4 - #256

Merged
github-actions[bot] merged 1 commit into
mainfrom
evals/run-31362802362-1
Aug 10, 2026
Merged

test(eval): record eval run at 1ad1ed4#256
github-actions[bot] merged 1 commit into
mainfrom
evals/run-31362802362-1

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor
3/3  112s  evals/check-before-acting.eval.ts / consults the docs before an unfamiliar task
3/3  49s  evals/check-before-acting.eval.ts / does not destructively act on an ambiguous request
3/3  13s  evals/check-before-acting.eval.ts / does not try to reassign a document to another type
3/3  24s  evals/configure-repositories.eval.ts / adds a locale
3/3  30s  evals/configure-repositories.eval.ts / creates an API token and makes the API private
3/3  31s  evals/configure-repositories.eval.ts / registers a webhook
3/3  25s  evals/configure-repositories.eval.ts / sets up a content preview
1/3  57s  evals/configure-repositories.eval.ts / updates previews for production after a deploy
2/3  111s  evals/configure-routes.eval.ts / routes a nested URL through a content relationship
0/3  4s  evals/configure-routes.eval.ts / routes non-default locales with an optional locale prefix
2/3  182s  evals/configure-routes.eval.ts / routes the home document to the root URL
2/3  134s  evals/design-sensible-models.eval.ts / handles a vague design request reasonably
0/3  52s  evals/design-sensible-models.eval.ts / models a page type with sensible field types
3/3  105s  evals/design-sensible-models.eval.ts / models a slice from a screenshot
3/3  51s  evals/design-sensible-models.eval.ts / models a testimonial slice from a text description
1/3  36s  evals/design-sensible-models.eval.ts / models a title as single-heading rich text and a social media handle as key text
3/3  122s  evals/design-sensible-models.eval.ts / models pages as page types and data as custom types
3/3  39s  evals/design-sensible-models.eval.ts / picks appropriate field types for a rating and a CTA
3/3  31s  evals/edit-models-precisely.eval.ts / adds a field
3/3  52s  evals/edit-models-precisely.eval.ts / adds a group field with nested fields
3/3  29s  evals/edit-models-precisely.eval.ts / adds a slice variation without losing existing ones
3/3  49s  evals/edit-models-precisely.eval.ts / adds constrained fields with the right config
3/3  29s  evals/edit-models-precisely.eval.ts / connects a slice to a page type
3/3  54s  evals/edit-models-precisely.eval.ts / extends a slice to match a design change
3/3  65s  evals/edit-models-precisely.eval.ts / is idempotent when the same task runs twice
3/3  29s  evals/edit-models-precisely.eval.ts / uses the documented config for a single-heading constraint
3/3  157s  evals/initialize-projects.eval.ts / adds Prismic to an existing Next.js app without clobbering it
0/3  32s  evals/initialize-projects.eval.ts / initializes Prismic in a Next.js project
3/3  35s  evals/initialize-projects.eval.ts / initializes with an existing repository
3/3  209s  evals/render-content.eval.ts / models and wires a landing page end to end
3/3  194s  evals/render-content.eval.ts / writes a slice component
3/3  27s  evals/sync-models.eval.ts / commits and pushes local model changes
2/3  38s  evals/sync-models.eval.ts / syncs in the right direction when the repo is newer

Run: https://github.com/prismicio/cli/actions/runs/31362802362


Note

Low Risk
Only updates tracked eval benchmark output; no runtime or CLI behavior changes in this PR.

Overview
Refreshes the committed full-suite agent eval snapshot in evals/results.json for model claude-sonnet-5 after CI run at commit 1ad1ed4.

Compared to the previous snapshot, several scenarios improve (e.g. sync-models push is now 3/3, nested route modeling 2/3, home-at-root routing 2/3, production preview update 1/3, locale add back to 3/3). Regressions or persistent gaps remain on Next.js init (0/3), blog post field typing (0/3), locale-prefixed routes (0/3), and partial passes on production previews and pull-when-repo-newer.

Each trial’s recorded CLI trace (calls, costUsd, durationS, pass) is updated to match this run; there is no application or eval-harness code change in the diff.

Reviewed by Cursor Bugbot for commit ffdaad5. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
github-actions Bot merged commit 450c117 into main Aug 10, 2026
@github-actions
github-actions Bot deleted the evals/run-31362802362-1 branch August 10, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants