// Module included in the following assemblies:
// * edge_computing/image-based-upgrade/ztp-image-based-upgrade.adoc

:_mod-docs-content-type: PROCEDURE
[id="ztp-image-based-upgrade-upgrade_{context}"]
= Moving to the Upgrade stage of the image-based upgrade with {lcao} and {ztp}

After you completed the `Prep` stage, you can upgrade the target cluster. During the upgrade process, the {oadp-short} Operator creates a backup of the artifacts specified in the {oadp-short} CRs, then the {lcao} upgrades the cluster.

If the upgrade fails or stops, an automatic rollback is initiated.
If you have an issue after the upgrade, you can initiate a manual rollback.
For more information about manual rollback, see "(Optional) Initiating a rollback with Lifecycle Agent and {ztp}".

.Prerequisites

* Complete the `Prep` stage.

.Procedure

. When you are ready to move to the `Upgrade` stage, create the `ClusterGroupUpgrade` CR on the target hub cluster that references the `Upgrade` policy:
+
[source,yaml]
----
apiVersion: ran.openshift.io/v1alpha1
kind: ClusterGroupUpgrade
metadata:
  name: cgu-ibu-upgrade
  namespace: default
spec:
  actions:
    beforeEnable:
      addClusterAnnotations:
        import.open-cluster-management.io/disable-auto-import: "true" <1>
    afterCompletion:
      removeClusterAnnotations:
      - import.open-cluster-management.io/disable-auto-import <2>
  clusters: 
  - spoke1
  enable: true
  managedPolicies: 
  - example-group-ibu-upgrade-stage-policy
  remediationStrategy: 
    canaries: 
      - spoke1
    maxConcurrency: 1 
    timeout: 240
----
<1> Applies the `disable-auto-import` annotation to the managed cluster before starting the upgrade. This annotation ensures the automatic importing of managed cluster is disabled during the upgrade stage until the cluster is ready.
<2> Removes the `disable-auto-import` annotation after the upgrade is complete.

. Apply the `Upgrade` policy by running the following command:
+
[source,terminal]
----
$ oc apply -f cgu-ibu-upgrade.yml
----

. Monitor the status by running the following command and wait for the `cgu-ibu-upgrade` `ClusterGroupUpgrade` to report `Completed`:
+
[source,terminal]
----
$ oc get cgu -n default
----

+
.Example output
[source,terminal]
----
NAME                              AGE   STATE       DETAILS
cgu-ibu-prep                      31h   Completed   All clusters are compliant with all the managed policies
cgu-ibu-upgrade                   31h   Completed   All clusters are compliant with all the managed policies
----

. When you are satisfied with the changes and ready to finalize the upgrade, create a `ClusterGroupUpgrade` CR on target hub cluster that references the policy that finalizes the upgrade:
+
[source,yaml]
----
apiVersion: ran.openshift.io/v1alpha1
kind: ClusterGroupUpgrade
metadata:
  name: cgu-ibu-finalize
  namespace: default
spec:
  actions:
    beforeEnable:
      removeClusterAnnotations:
      - import.open-cluster-management.io/disable-auto-import
  clusters: 
  - spoke1
  enable: true
  managedPolicies: 
  - example-group-ibu-finalize-stage-policy
  remediationStrategy: 
    canaries: 
      - spoke1
    maxConcurrency: 1 
    timeout: 240
----

+
[IMPORTANT]
====
Ensure that no other `ClusterGroupUpgrade` CRs are in progress because this causes {cgu-operator} to continuously reconcile them. Delete all `"In-Progress"` `ClusterGroupUpgrade` CRs before applying the `cgu-ibu-finalize.yaml`. 
====

. Apply the policy by running the following command:
+
[source,terminal]
----
$ oc apply -f cgu-ibu-finalize.yaml
----

. Monitor the status and wait for the `cgu-ibu-finalize` `ClusterGroupUpgrade` to report `Completed` by running the following command: 
+
[source,terminal]
----
$ oc get cgu -n default
----

+
.Example output
[source,terminal]
----
NAME                    AGE   STATE       DETAILS
cgu-ibu-finalize        30h   Completed   All clusters are compliant with all the managed policies
cgu-ibu-prep            31h   Completed   All clusters are compliant with all the managed policies
cgu-ibu-upgrade         31h   Completed   All clusters are compliant with all the managed policies
----

. You can remove the {oadp-short} Operator and its configuration files after a successful upgrade.

.. Change the `complianceType` to `mustnothave` for the {oadp-short} Operator namespace, Operator group, and subscription in the `common-ranGen.yaml` file.
+
[source,yaml]
----
[...]
- fileName: OadpSubscriptionNS.yaml
  policyName: "subscriptions-policy"
  complianceType: mustnothave
- fileName: OadpSubscriptionOperGroup.yaml
  policyName: "subscriptions-policy"
  complianceType: mustnothave
- fileName: OadpSubscription.yaml
  policyName: "subscriptions-policy"
  complianceType: mustnothave
- fileName: OadpOperatorStatus.yaml
  policyName: "subscriptions-policy"
  complianceType: mustnothave
[...]
----

.. Change the `complianceType` to `mustnothave` for the {oadp-short} Operator namespace, Operator group, and subscription in the site `PolicyGenTemplate` file.
+
[source,yaml]
----
- fileName: OadpSecret.yaml
  policyName: "config-policy"
  complianceType: mustnothave
- fileName: OadpBackupStorageLocationStatus.yaml
  policyName: "config-policy"
  complianceType: mustnothave
- fileName: DataProtectionApplication.yaml
  policyName: "config-policy"
  complianceType: mustnothave
----

.. Merge the changes with your custom site repository and wait for the ArgoCD application to synchronize the change to the hub cluster. The status of the `common-subscriptions-policy` and the `example-cnf-config-policy` policies change to `Non-Compliant`.

.. Apply the change to your target clusters by using the {cgu-operator-full}. For more information about rolling out configuration changes, see "Update policies on managed clusters".

.. Monitor the process. When the status of the `common-subscriptions-policy` and the `example-cnf-config-policy` policies for a target cluster are `Compliant`, the {oadp-short} Operator has been removed from the cluster. Get the status of the policies by running the following commands:
+
--
[source,terminal]
----
$ oc get policy -n ztp-common common-subscriptions-policy
----

[source,terminal]
----
$ oc get policy -n ztp-site example-cnf-config-policy
----
--

.. Delete the {oadp-short} Operator namespace, Operator group and subscription, and configuration CRs from `spec.sourceFiles` in the `common-ranGen.yaml` and the site `PolicyGenTemplate` files.

.. Merge the changes with your custom site repository and wait for the ArgoCD application to synchronize the change to the hub cluster. The policy remains compliant.