Error installing on Azure Kubernetes Service

<!-- Make sure that you visit our User Guide at https://kubevirt.io/user-guide. --> **What happened**: I have followed the given installation steps in the installation guide but the pods in the 'kubevirt' namespace are stuck in "CrashLoopBack". The installation steps are: ```bash export VERSION=$(curl -s https://storage.googleapis.com/kubevirt-prow/release/kubevirt/kubevirt/stable.txt) kubectl create -f "https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml" kubectl create -f "https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml" ``` The output of the logs of the "virt-operator" pods is: ``` W1109 13:52:17.924128 1 client_config.go:659] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. W1109 13:52:17.924453 1 client_config.go:659] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. {"component":"virt-operator","level":"info","msg":"we are on kubernetes","pos":"application.go:241","timestamp":"2024-11-09T13:52:17.952873Z"} {"component":"virt-operator","level":"info","msg":"servicemonitor is not defined","pos":"application.go:259","timestamp":"2024-11-09T13:52:17.958575Z"} {"component":"virt-operator","level":"info","msg":"prometheusrule is not defined","pos":"application.go:274","timestamp":"2024-11-09T13:52:17.965930Z"} {"component":"virt-operator","level":"info","msg":"validatingAdmissionPolicyBindingEnabled is not defined","pos":"application.go:289","timestamp":"2024-11-09T13:52:17.973928Z"} {"component":"virt-operator","level":"info","msg":"validatingAdmissionPolicyEnabled is not defined","pos":"application.go:304","timestamp":"2024-11-09T13:52:17.979419Z"} {"component":"virt-operator","level":"info","msg":"Operator image: quay.io/kubevirt/virt-operator:v1.3.1","pos":"application.go:321","timestamp":"2024-11-09T13:52:17.980380Z"} {"component":"virt-operator","level":"info","msg":"set log verbosity to 2","pos":"application.go:533","timestamp":"2024-11-09T13:52:17.980469Z"} {"component":"virt-operator","level":"info","msg":"set log verbosity to 2","pos":"application.go:533","timestamp":"2024-11-09T13:52:17.980494Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorCRDInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.980704Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorServiceInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.980890Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorServiceMonitor","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.980917Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorClusterRoleInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981008Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorClusterRoleBindingInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981084Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorRoleBindingInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981109Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer operatorMutatingWebhookInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981121Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer secretsInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981134Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorConfigMapInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981145Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer vmClusterInstancetypeInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981161Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer kubeVirtInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981180Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer installStrategyJobsInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981190Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer operatorAPIServiceInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981202Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer installStrategyConfigMapInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981213Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer operatorPodDisruptionBudgetInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981230Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer namespaceInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981240Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorRoleInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981255Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorDeploymentInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981267Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer operatorPodsInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981277Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer CRDInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981292Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorDaemonSetInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981304Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer vmClusterPreferenceInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981322Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorRoute","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981333Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorPrometheusRuleInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981345Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorValidatingAdmissionPolicyInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981362Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer OperatorServiceAccountInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981372Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer operatorValidatingWebhookInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981385Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorSCC","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981396Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer FakeOperatorValidatingAdmissionPolicyBindingInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981410Z"} {"component":"virt-operator","level":"info","msg":"STARTING informer extensionsConfigMapInformer","pos":"virtinformers.go:363","timestamp":"2024-11-09T13:52:17.981460Z"} {"component":"virt-operator","level":"info","msg":"set log verbosity to 2","pos":"application.go:533","timestamp":"2024-11-09T13:52:17.990076Z"} W1109 13:52:17.995330 1 reflector.go:547] pkg/controller/virtinformers.go:364: failed to list *v1beta1.VirtualMachineClusterInstancetype: the server could not find the requested resource (get virtualmachineclusterinstancetypes.instancetype.kubevirt.io) E1109 13:52:17.995441 1 reflector.go:150] pkg/controller/virtinformers.go:364: Failed to watch *v1beta1.VirtualMachineClusterInstancetype: failed to list *v1beta1.VirtualMachineClusterInstancetype: the server could not find the requested resource (get virtualmachineclusterinstancetypes.instancetype.kubevirt.io) W1109 13:52:17.995706 1 reflector.go:547] pkg/controller/virtinformers.go:364: failed to list *v1beta1.VirtualMachineClusterPreference: the server could not find the requested resource (get virtualmachineclusterpreferences.instancetype.kubevirt.io) E1109 13:52:17.995824 1 reflector.go:150] pkg/controller/virtinformers.go:364: Failed to watch *v1beta1.VirtualMachineClusterPreference: failed to list *v1beta1.VirtualMachineClusterPreference: the server could not find the requested resource (get virtualmachineclusterpreferences.instancetype.kubevirt.io) {"component":"virt-operator","level":"info","msg":"set log verbosity to 2","pos":"application.go:533","timestamp":"2024-11-09T13:52:18.164122Z"} {"component":"virt-operator","level":"info","msg":"Attempting to acquire leader status","pos":"application.go:469","timestamp":"2024-11-09T13:52:18.181823Z"} I1109 13:52:18.181885 1 leaderelection.go:250] attempting to acquire leader lease kubevirt/virt-operator... {"component":"virt-operator","level":"info","msg":"set log verbosity to 2","pos":"application.go:533","timestamp":"2024-11-09T13:52:18.182042Z"} {"component":"virt-operator","level":"info","msg":"Reinitialize virt-operator, ServiceMonitor has been introduced","pos":"application.go:484","timestamp":"2024-11-09T13:52:18.182175Z"} ``` The output of `kubectl get crds | grep "kubevirt"` is ``` kubevirts.kubevirt.io 2024-11-09T14:37:33Z ``` The output of `kubectl get apiservices` is : ``` NAME SERVICE AVAILABLE AGE v1. Local True 119m v1.admissionregistration.k8s.io Local True 119m v1.apiextensions.k8s.io Local True 119m v1.apps Local True 119m v1.authentication.k8s.io Local True 119m v1.authorization.k8s.io Local True 119m v1.autoscaling Local True 119m v1.azmonitoring.coreos.com Local True 108m v1.batch Local True 119m v1.certificates.k8s.io Local True 119m v1.coordination.k8s.io Local True 119m v1.crd.projectcalico.org Local True 119m v1.discovery.k8s.io Local True 119m v1.eraser.sh Local True 117m v1.events.k8s.io Local True 119m v1.flowcontrol.apiserver.k8s.io Local True 119m v1.kubevirt.io Local True 101s v1.networking.k8s.io Local True 119m v1.node.k8s.io Local True 119m v1.operator.tigera.io Local True 119m v1.policy Local True 119m v1.rbac.authorization.k8s.io Local True 119m v1.scheduling.k8s.io Local True 119m v1.snapshot.storage.k8s.io Local True 119m v1.storage.k8s.io Local True 119m v1alpha1.eraser.sh Local True 117m v1alpha3.kubevirt.io Local True 101s v1beta1.admissionregistration.k8s.io Local True 119m v1beta1.authentication.k8s.io Local True 119m v1beta1.metrics.k8s.io kube-system/metrics-server True 119m v1beta3.flowcontrol.apiserver.k8s.io Local True 119m v2.autoscaling Local True 119m ``` **What you expected to happen**: The usual deployment of Kubernetes as mentioned in the docs. **How to reproduce it (as minimally and precisely as possible)**: 1. Create a AKS cluster with Kubernetes version v1.29 2. Follow Kubevirt installation steps **Additional context**: It works fine with self-managed Rancher k3s (Kubernetes v1.26) **Environment**: - KubeVirt version (use `virtctl version`): v1.3.1 - Kubernetes version (use `kubectl version`): v1.29.0 - VM or VMI specifications: N/A - Cloud provider or hardware configuration: Azure (Azure Kubernetes Service) - OS (e.g. from /etc/os-release): Linux - Kernel (e.g. `uname -a`): N/A - Install tools: N/A - Others: N/A