Skip to content

fix(stardust): migrate eval criteria.json to tessl weighted_checklist… - #267

Draft
mandeep294 wants to merge 1 commit into
adobe:mainfrom
mandeep294:fix/stardust-criteria-schema
Draft

fix(stardust): migrate eval criteria.json to tessl weighted_checklist…#267
mandeep294 wants to merge 1 commit into
adobe:mainfrom
mandeep294:fix/stardust-criteria-schema

Conversation

@mandeep294

@mandeep294 mandeep294 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

… schema

The Release Skills workflow's tessl plugin publish step began enforcing the new criteria.json schema between 2026-07-04 and 2026-07-07, failing the stardust plugin publish with:

Invalid criteria.json (in migrate-self-contained-bundle):
context: expected string, received undefined
type: expected "weighted_checklist"
checklist: expected array, received undefined

Failed actions:
https://github.com/adobe/skills/actions/runs/28875097726/job/85647927305

All 9 stardust eval criteria.json still used the legacy {name, total, criteria:[{id, weight, description}]} shape. Convert them to the {context, type: "weighted_checklist", checklist:[{name, max_score, description}]} shape the other Adobe-skills plugins already use.

  • id -> name (identifier preserved, lossless)
  • weight -> max_score (sums to 100 per eval, unchanged)
  • description kept verbatim
  • context authored per eval from the README coverage table
  • README Format/Running sections updated to describe the new shape

Item counts and weight sums verified unchanged for all 9 evals.

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

… schema

The Release Skills workflow's `tessl plugin publish` step began enforcing
the new criteria.json schema between 2026-07-04 and 2026-07-07, failing
the stardust plugin publish with:

  Invalid criteria.json (in migrate-self-contained-bundle):
    context: expected string, received undefined
    type:    expected "weighted_checklist"
    checklist: expected array, received undefined

All 9 stardust eval criteria.json still used the legacy
{name, total, criteria:[{id, weight, description}]} shape. Convert them
to the {context, type: "weighted_checklist", checklist:[{name, max_score,
description}]} shape the other Adobe-skills plugins already use.

- id -> name (identifier preserved, lossless)
- weight -> max_score (sums to 100 per eval, unchanged)
- description kept verbatim
- context authored per eval from the README coverage table
- README Format/Running sections updated to describe the new shape

Item counts and weight sums verified unchanged for all 9 evals.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant