Skip to content

feat: support optional skills packages - #1726

Open
scottt732 wants to merge 3 commits into
kelos-dev:mainfrom
scottt732:fix/optional-skills-install
Open

feat: support optional skills packages#1726
scottt732 wants to merge 3 commits into
kelos-dev:mainfrom
scottt732:fix/optional-skills-install

Conversation

@scottt732

@scottt732 scottt732 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind api

What this PR does / why we need it:

Adds an opt-in spec.skills[].optional field to the v1alpha2 AgentConfig API.

Required packages keep the existing fail-fast behavior. An optional package logs its source and allows the remaining packages and agent startup to continue when npx skills add fails. This prevents an unavailable third-party skills package from blocking an otherwise usable agent when the caller has explicitly accepted degraded capability.

The field defaults to false, is preserved across v1alpha1 conversion through an annotation, and does not add new surface area to the compatibility API version.

Which issue(s) this PR is related to:

N/A

Special notes for your reviewer:

  • make verify passes.
  • make test passes.
  • make build passes after removing a Homebrew LDFLAGS value from the local environment; the Makefile forwards that variable to Go's -ldflags.
  • Script-execution tests cover optional failure with a later successful package, required fail-fast behavior, and the all-optional/all-failed case.
  • Conversion tests cover round-trip preservation and ambiguous duplicate protection.

Does this PR introduce a user-facing change?

AgentConfig v1alpha2 skills can set `optional: true` so a package installation failure is logged without blocking agent startup.

Summary by cubic

Adds spec.skills[].optional to the v1alpha2 AgentConfig API so agent startup continues when an optional skills package fails to install, while required packages keep the existing fail-fast behavior.

  • Optional package failures are logged with the package source; remaining packages and agent startup proceed, and an all-optional failure leaves an empty plugin skills directory.
  • Required packages now fail fast on install or relocation failures; the field defaults to false and is preserved across v1alpha1 conversion via an annotation.
  • Script-execution tests cover optional failure with later success, required fail-fast behavior, and all-optional/all-failed cases.

Written for commit 8d4bf5a. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/controller/job_builder.go">

<violation number="1" location="internal/controller/job_builder.go:1406">
P2: When a required `npx skills add` succeeds after leaving an empty `.agents/skills`, this check passes and the agent starts without the required skills. Require at least one child entry before proceeding.</violation>
</file>

<file name="internal/conversion/agentconfig_test.go">

<violation number="1" location="internal/conversion/agentconfig_test.go:178">
P3: The new optional round-trip test only asserts the optional flag survives. The sibling TestAgentConfigRoundTrip_PreservesSkillsSecretRef also verifies the preservation annotation is removed from the hub after restore and not mutated on the source spoke; the optional test omits both checks, so a regression where the annotation leaks into the hub object would go undetected. Add assertions that hub.Annotations no longer contains preservedSkillsOptionalAnnotation after restore.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

lines = append(lines, fmt.Sprintf("mkdir -p %s", shellQuote(pluginSkillsDir)))
if hasRequiredSkills {
lines = append(lines,
fmt.Sprintf("[ -d %s ] || { echo 'No skills.sh skills were installed' >&2; exit 1; }", shellQuote(installDir)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When a required npx skills add succeeds after leaving an empty .agents/skills, this check passes and the agent starts without the required skills. Require at least one child entry before proceeding.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At internal/controller/job_builder.go, line 1406:

<comment>When a required `npx skills add` succeeds after leaving an empty `.agents/skills`, this check passes and the agent starts without the required skills. Require at least one child entry before proceeding.</comment>

<file context>
@@ -1390,10 +1400,16 @@ func buildSkillsInstallScript(skills []kelos.SkillsShSpec, authEnvs []skillsAuth
+	lines = append(lines, fmt.Sprintf("mkdir -p %s", shellQuote(pluginSkillsDir)))
+	if hasRequiredSkills {
+		lines = append(lines,
+			fmt.Sprintf("[ -d %s ] || { echo 'No skills.sh skills were installed' >&2; exit 1; }", shellQuote(installDir)),
+		)
+	}
</file context>
Suggested change
fmt.Sprintf("[ -d %s ] || { echo 'No skills.sh skills were installed' >&2; exit 1; }", shellQuote(installDir)),
fmt.Sprintf("[ -d %s ] && [ -n \"$(find %s -mindepth 1 -maxdepth 1 -print -quit)\" ] || { echo 'No skills.sh skills were installed' >&2; exit 1; }", shellQuote(installDir), shellQuote(installDir)),

t.Fatalf("agentConfigToHub() error = %v", err)
}

if len(hub.Spec.Skills) != 1 || !hub.Spec.Skills[0].Optional {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The new optional round-trip test only asserts the optional flag survives. The sibling TestAgentConfigRoundTrip_PreservesSkillsSecretRef also verifies the preservation annotation is removed from the hub after restore and not mutated on the source spoke; the optional test omits both checks, so a regression where the annotation leaks into the hub object would go undetected. Add assertions that hub.Annotations no longer contains preservedSkillsOptionalAnnotation after restore.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At internal/conversion/agentconfig_test.go, line 178:

<comment>The new optional round-trip test only asserts the optional flag survives. The sibling TestAgentConfigRoundTrip_PreservesSkillsSecretRef also verifies the preservation annotation is removed from the hub after restore and not mutated on the source spoke; the optional test omits both checks, so a regression where the annotation leaks into the hub object would go undetected. Add assertions that hub.Annotations no longer contains preservedSkillsOptionalAnnotation after restore.</comment>

<file context>
@@ -158,6 +158,55 @@ func TestAgentConfigRoundTrip_PreservesSkillsSecretRef(t *testing.T) {
+		t.Fatalf("agentConfigToHub() error = %v", err)
+	}
+
+	if len(hub.Spec.Skills) != 1 || !hub.Spec.Skills[0].Optional {
+		t.Fatalf("Skills = %#v, want one optional skill", hub.Spec.Skills)
+	}
</file context>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/controller/job_builder.go">

<violation number="1" location="internal/controller/job_builder.go:1411">
P3: The `for skill_path in <installDir>/*` glob replaces `find -mindepth 1 -maxdepth 1`, but `*` in POSIX sh does not match hidden (dot-prefixed) entries. Any top-level dotfile/directory in `$HOME/.agents/skills` is skipped and then removed by the subsequent `rm -rf`, silently dropping skill content that the previous `find` command relocated. If skills.sh writes a top-level hidden entry, its contents are lost. Include hidden entries (e.g. a `.[!.]*` glob) or keep the `find` form.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

fmt.Sprintf("mkdir -p %s", shellQuote(pluginSkillsDir)),
fmt.Sprintf("mv %s/* %s/", shellQuote(installDir), shellQuote(pluginSkillsDir)),
fmt.Sprintf("if [ -d %s ]; then", shellQuote(installDir)),
fmt.Sprintf(" for skill_path in %s/*; do", shellQuote(installDir)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The for skill_path in <installDir>/* glob replaces find -mindepth 1 -maxdepth 1, but * in POSIX sh does not match hidden (dot-prefixed) entries. Any top-level dotfile/directory in $HOME/.agents/skills is skipped and then removed by the subsequent rm -rf, silently dropping skill content that the previous find command relocated. If skills.sh writes a top-level hidden entry, its contents are lost. Include hidden entries (e.g. a .[!.]* glob) or keep the find form.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At internal/controller/job_builder.go, line 1411:

<comment>The `for skill_path in <installDir>/*` glob replaces `find -mindepth 1 -maxdepth 1`, but `*` in POSIX sh does not match hidden (dot-prefixed) entries. Any top-level dotfile/directory in `$HOME/.agents/skills` is skipped and then removed by the subsequent `rm -rf`, silently dropping skill content that the previous `find` command relocated. If skills.sh writes a top-level hidden entry, its contents are lost. Include hidden entries (e.g. a `.[!.]*` glob) or keep the `find` form.</comment>

<file context>
@@ -1408,7 +1408,10 @@ func buildSkillsInstallScript(skills []kelos.SkillsShSpec, authEnvs []skillsAuth
 	lines = append(lines,
 		fmt.Sprintf("if [ -d %s ]; then", shellQuote(installDir)),
-		fmt.Sprintf("  find %s -mindepth 1 -maxdepth 1 -exec mv {} %s/ \\;", shellQuote(installDir), shellQuote(pluginSkillsDir)),
+		fmt.Sprintf("  for skill_path in %s/*; do", shellQuote(installDir)),
+		"    [ -e \"$skill_path\" ] || continue",
+		fmt.Sprintf("    mv \"$skill_path\" %s/", shellQuote(pluginSkillsDir)),
</file context>
Suggested change
fmt.Sprintf(" for skill_path in %s/*; do", shellQuote(installDir)),
fmt.Sprintf(" for skill_path in %s/* %s/.[!.]*; do", shellQuote(installDir), shellQuote(installDir)),

@scottt732

Copy link
Copy Markdown
Contributor Author

Gentle nudge on this one — it has been in needs-triage/needs-actor since Sep 3 and has not picked up a reviewer yet.

Current state, in case it helps triage:

  • MERGEABLE, no conflicts against main
  • make verify, make test, and make build all pass
  • Tests cover the three cases that matter: an optional failure followed by a successful package, required fail-fast, and the all-optional/all-failed case
  • Conversion tests cover v1alpha1 round-trip preservation and ambiguous-duplicate protection

The field defaults to false and round-trips through v1alpha1 via an annotation, so it is inert for anyone not opting in and adds no surface area to the compatibility API version.

Happy to rebase, split the conversion change into its own PR, or narrow the scope if any of that would make it easier to review.

@gjkim42

gjkim42 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

/kelos claude-api-review

@kelos-bot

kelos-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🤖 Kelos Task Status

Task kelos-claude-api-reviewer-issue-comment-16ea3782b289 has succeeded. ✅

@gjkim42

gjkim42 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Looks good to me.
Can you add an e2e test for this?

@gjkim42

gjkim42 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

/kelos claude-review

@kelos-bot

kelos-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🤖 Kelos Task Status

Task kelos-claude-reviewer-issue-comment-2d85ee4cc183 has succeeded. ✅

@kelos-bot

kelos-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🤖 Kelos Claude API Reviewer Agent @gjkim42

API Design Review

Verdict: APPROVE
Scope: Adds spec.skills[].optional (bool, defaults to false) to the v1alpha2 AgentConfig SkillsShSpec, preserves it across v1alpha1 conversion via the kelos.dev/v1alpha2-skills-optional annotation, and regenerates both CRD copies plus docs/reference.md. Reviewed at head 8d4bf5a.

No blocking issues. The field is additive, correctly scoped to the storage version, and follows existing Kelos and upstream naming. Two documentation/contract clarifications are worth landing before merge; everything else is non-blocking.

Findings

Compatibility (no issues)

  • api/v1alpha2/agentconfig_types.go:87-90 — Additive optional bool with omitempty and +optional. No existing field changes kind, no validation is tightened, and false/absent are equivalent, so every existing AgentConfig still applies. examples/ and self-development/ need no sweep.
  • internal/manifests/install-crd.yaml:614 and the chart copy — The new property appears only under the v1alpha2 schema; the served v1alpha1 schema is untouched, matching the "latest version only" rule. Both generated CRD copies are updated and the CI verify job passes on this head, so make update artifacts are complete (no deepcopy change is needed for a bool, and pkg/generated has no apply-configurations to regenerate).

Documentation (P2, non-blocking but please fix)

  • api/v1alpha2/agentconfig_types.go:87-88 and docs/reference.md:728 — The godoc and reference row say startup continues "when this package cannot be installed", but optional only relaxes the skills-install init-container step. validateSkillsAuthSecrets (internal/controller/task_controller.go:499-528, called from task_controller.go:372 and session_controller.go:1382) does not consult Optional, so a missing Secret or missing/empty GITHUB_TOKEN on an optional package still fails the Task before the Job is created. That fail-fast behavior is the right call for a configuration error, but the contract should say so. Suggested godoc:
    // Optional, when true, logs an installation failure for this package and
    // continues installing the remaining packages instead of failing the Task.
    // Defaults to false. SecretRef validation is not affected: a missing or
    // invalid Secret still fails the Task before the Job is created.
    and append a matching sentence to the reference row.
  • api/v1alpha2/agentconfig_types.go:88 — "Required packages remain fail-fast" refers to a concept with no field behind it. State the default explicitly instead, the way AllowInsecure does (taskspawner_types.go:822), and use the "X, when true, ..." phrasing already used by MentionOptional (taskspawner_types.go:705). The snippet above covers both.

Naming and shape (P3, no change needed)

  • optional matches upstream corev1 usage on SecretKeySelector, ConfigMapKeySelector, and EnvFromSource ("tolerate absence") and Kelos's own mentionOptional, so the name will age well. A failurePolicy enum was the alternative; with no concrete caller for retries or a Task condition today, the bool is the minimal surface and the right choice now. If a richer policy is ever needed it would be a new field rather than a new enum value, which is acceptable given the precedent.
  • Plain bool rather than *bool is fine here because unset and false mean the same thing, consistent with allowInsecure.

Conversion (P3)

  • internal/conversion/agentconfig.go:311-355 — Mirrors the existing secretRef preservation pattern (index + source + skill match, ambiguity guard, annotation stripped on the hub, malformed annotation ignored). Correct for the multi-version contract.
  • internal/conversion/agentconfig_test.go:161-181 — The optional round-trip test asserts only that the flag survives. The sibling secretRef test (agentconfig_test.go:149-154) also checks that the hub no longer carries the preservation annotation and that the spoke annotation is not mutated during restore. Adding the same two assertions keeps the two annotations' contracts pinned identically.

Suggestions (optional)

  • e2e coverage for the documented contract. The "startup continues" promise is verified only at the shell-script level. Entrypoints pass every plugin directory through (claude-code/kelos_entrypoint.sh:88-89 via --plugin-dir, codex/kelos_entrypoint.sh:37-41 via the copy loop), and the all-optional-failed case leaves an empty skills-sh/skills directory. An e2e Task with one optional: true unreachable source plus the existing kelos-dev/e2e-skills fixture, asserting Succeeded and the marker string, would prove the contract end to end. This aligns with the maintainer's request above.
  • Observability follow-up. An optional failure is visible only in the skills-install init-container log. That is fine for the minimal API; if operators later need it, a Task event or condition would be the natural next step and does not require an API change now.
  • CLI parity follow-up. kelos create agentconfig --skills-sh source:skill (internal/cli/create_agentconfig.go:132) has no way to set optional. Not required for this PR, but worth tracking so the CLI does not lag the API.

/kelos needs-input

@kelos-bot

kelos-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🤖 Kelos Claude Reviewer Agent @gjkim42

Review Summary

Verdict: APPROVE
Overall correctness: patch is correct
Scope: Adds spec.skills[].optional to the v1alpha2 AgentConfig, makes the skills-install init script log and continue when an optional package fails while required packages stay fail-fast, and preserves the flag across v1alpha1 conversion via an annotation. Reviewed at head 8d4bf5a.

Findings Overview

Priority Count File:Line Summary
P0 0 none
P1 0 none
P2 2 internal/cli/agentconfig_compat.go:98 agentConfigFitsV1alpha1 ignores Optional, so the v1alpha1 fallback creates an object on which the flag is not honored
P2 test/e2e/skills_test.go:77 No e2e coverage for the "startup continues" contract (requested by maintainer after the last commit)
P3 2 internal/controller/job_builder.go:1406 Required-package guard no longer fails when the install dir exists but is empty
P3 internal/conversion/agentconfig_test.go:161 Round-trip test omits the annotation-cleanup assertions its secretRef sibling has

Findings

Correctness

  • [P2] internal/cli/agentconfig_compat.go:98agentConfigFitsV1alpha1 does not treat Optional: true as a v1alpha2-only capability. On a cluster that serves only v1alpha1 CRDs, createAgentConfig (internal/cli/create_agentconfig.go:120) converts the spec and creates it as v1alpha1; the stored object then carries only the kelos.dev/v1alpha2-skills-optional annotation, which nothing on that cluster honors, so the package is installed as required and an install failure blocks the agent despite the caller's opt-in. The commit that introduced secretRef preservation (6ef2305) added its check to this helper plus TestCreateAgentConfigRejectsSkillsSecretRefWithoutV1alpha2 in the same change; do the same for Optional and extend the reason string at agentconfig_compat.go:72 (currently "MCP env valueFrom or skills secretRef"). This is unreachable from today's --skills-sh flag syntax, so it is latent, but the helper's contract is wrong as of this PR.

Tests

  • [P2] test/e2e/skills_test.go:77 — gjkim42 asked for an e2e test today, after the last commit on Sep 3; the PR has none. The "startup continues" contract is currently verified only by running the generated script under sh with a stubbed npx. Since --skills-sh cannot set optional, create the AgentConfig through f.KelosClientset.ApiV1alpha2().AgentConfigs(ns).Create (as the authenticated case already does at skills_test.go:162) with the existing kelos-dev/e2e-skills:kelos-e2e required entry plus an optional: true entry pointing at a nonexistent repo, keep the verify-skills-install init container and the KELOS_E2E_SKILL_MARKER_x7k2p9 log assertion, and assert the Task reaches Succeeded. A second case with only the unreachable optional package asserting Succeeded would pin the empty-plugin-dir path through the real entrypoint.
  • [P3] internal/conversion/agentconfig_test.go:161TestAgentConfigRoundTrip_PreservesOptionalSkill asserts only that Optional survives. The sibling TestAgentConfigRoundTrip_PreservesSkillsSecretRef (lines 149-154) also asserts that the hub no longer carries the preservation annotation after restore and that the spoke annotation is untouched; without those, a regression that leaks kelos.dev/v1alpha2-skills-optional onto hub objects passes. Add the same two assertions for the new annotation.

Correctness (low)

  • [P3] internal/controller/job_builder.go:1406 — The required-package guard now checks only that .agents/skills exists. Before this PR, mv '<installDir>'/* … under set -e also failed when the directory existed but was empty, because the unmatched glob was passed literally to mv. With the tolerant loop at lines 1411-1412, a required package whose npx skills add exits 0 without producing a skill (for example a -s name absent from the package, if the CLI exits 0 in that case) now passes the "No skills.sh skills were installed" guard and the agent starts without the required skill. Counting moved entries in the loop and failing when hasRequiredSkills and the count is zero restores the previous strictness and can replace the [ -d ] check.

Key takeaways

  • Script semantics are sound: optional packages run under if ! …; then, which set -e does not trip, while required packages stay bare commands so the first failure exits. All four entrypoints tolerate an empty skills-sh/skills directory ([ -d ] || continue loops in claude-code, codex, and cursor; cp -r of an empty dir in gemini), so the all-optional-failed case starts the agent as intended.
  • The API review at this same head flagged the godoc and docs/reference.md wording: optional does not relax validateSkillsAuthSecrets (internal/controller/task_controller.go:499), so a missing or empty Secret still fails the Task before Job creation. That is still unaddressed at 8d4bf5a and is not repeated as a separate finding here.
  • Both generated CRD copies and docs/reference.md are updated; a bool needs no deepcopy change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/api Categorizes issue or PR as related to API changes needs-actor needs-priority needs-triage release-note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants