Upgrade from 0.8.5 to 0.9.5

[Archana Anand (ArchanaAnand0212)]

Hi
I am trying to upgrade from v0.8.5 to v0.9.5 and seeing the error

[ ~/fleet ]$ git describe --tags
v0.9.5

"pod": k8s-hub-agent-67457568b8-k6n4d,
"container": k8s-hub-agent,
"log": E0213 00:02:49.894279       1 rollout/controller.go:134] "Failed to find all matching overrides for the clusterResourcePlacement" err="failed to get API group resources: unable to retrieve the complete list of server APIs: placement.kubernetes-fleet.io/v1alpha1: the server could not find the requested resource" clusterResourcePlacement="aranand-testing-upgrade-0-9-5"

The CRP resource created is

apiVersion: placement.kubernetes-fleet.io/v1beta1
kind: ClusterResourcePlacement
metadata:
  creationTimestamp: "2025-02-12T22:54:07Z"
  finalizers:
  - kubernetes-fleet.io/crp-cleanup
  - kubernetes-fleet.io/scheduler-cleanup
  generation: 2
  labels:
   dummy: aranand-testing-upgrade-0-9-5
  name: aranand-testing-upgrade-0-9-5
  ownerReferences:
  - apiVersion: tenant.linkedin.com/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: NamespaceConfig
    name: aranand-testing-upgrade-0-9-5
    uid: e5755a8d-1eb9-4b2d-9455-a97c4c1c5686
  resourceVersion: "1386028025"
  uid: 12a8470b-4cd1-4d4b-ba23-96c2dc707bf3
spec:
  policy:
    clusterNames:
    - dummy-member-cluster
    placementType: PickFixed
  resourceSelectors:
  - group: ""
    kind: Namespace
    name: aranand-testing-upgrade-0-9-5
    version: v1
  revisionHistoryLimit: 10
  strategy:
    type: RollingUpdate
status:
  conditions:
  - lastTransitionTime: "2025-02-12T23:21:25Z"
    message: found all the clusters needed as specified by the scheduling policy
    observedGeneration: 2
    reason: SchedulingPolicyFulfilled
    status: "True"
    type: ClusterResourcePlacementScheduled
  - lastTransitionTime: "2025-02-13T00:19:23Z"
    message: All 1 cluster(s) are synchronized to the latest resources on the hub
      cluster
    observedGeneration: 2
    reason: SynchronizeSucceeded
    status: "True"
    type: ClusterResourcePlacementSynchronized
  - lastTransitionTime: "2025-02-12T22:56:43Z"
    message: 'Works need to be synchronized on the hub cluster or there are still
      manifests pending to be processed by the 1 member clusters '
    observedGeneration: 2
    reason: ApplyPending
    status: Unknown
    type: ClusterResourcePlacementApplied
  placementStatuses:
  - clusterName: dummy-member-cluster
    conditions:
    - lastTransitionTime: "2025-02-12T23:21:25Z"
      message: 'Successfully scheduled resources for placement in dummy-member-cluster: picked
        by scheduling policy'
      observedGeneration: 2
      reason: ScheduleSucceeded
      status: "True"
      type: ResourceScheduled
    - lastTransitionTime: "2025-02-13T00:19:23Z"
      message: Successfully Synchronized work(s) for placement
      observedGeneration: 2
      reason: WorkSynchronizeSucceeded
      status: "True"
      type: WorkSynchronized
    - lastTransitionTime: "2025-02-12T23:21:25Z"
      message: Works need to be synchronized on the hub cluster or there are still
        manifests pending to be processed by the member cluster
      observedGeneration: 2
      reason: ApplyPending
      status: Unknown
      type: ResourceApplied
  selectedResources:
  - group: rbac.authorization.k8s.io
    kind: RoleBinding
    name: app-edit
    namespace: aranand-testing-upgrade-0-9-5
    version: v1
  - kind: Namespace
    name: aranand-testing-upgrade-0-9-5
    version: v1

Error is probably preventing from creation of Works
What would be the suggestion here so we don't break existing changes in v0.8.5? Thanks

[Chen Yu (michaelawyu)]

Hi Archana! It seems that one of the APIs required by v0.9.5 was not installed on the Fleet hub cluster; if possible, would you mind running the command kubectl api-resources and pasting the output here for further analysis? I am guessing that currently the ResourceOverride and ClusterResourceOverride APIs are not installed yet (i.e., the agent image is bumped, but the new APIs are not added).

If so, to fix the issue, you might need to apply the CRDs here again. You may also upgrade the helm chart to have the CRDs applied.

[Archana Anand (ArchanaAnand0212)]

Thanks Chen Yu (@michaelawyu) for looking into these.
I got little confused by the error message, you are right that it is for the missing CRD, CRO and RO.