Creating Windows Images Based on ISO using KubeVirt

This document discusses a virtual machine solution based on the open-source component KubeVirt, using KubeVirt virtualization technology to create a Windows operating system image through an ISO image file. The ISO files are uploaded to the cluster via CDI DataVolume, eliminating the need to build and push container images to a registry.

Prerequisites

  • All components in the cluster are functioning correctly.

  • Please prepare the Windows image and the latest virtio-win-tools in advance.

  • The kubectl command-line tool is installed and configured to access the cluster.

  • A StorageClass that supports ReadWriteOnce (RWO) access mode is available in the cluster.

Constraints and Limitations

  • When starting KubeVirt, the size of the custom image's filesystem will affect the speed of writing the image to the disk in PVC. If the filesystem is too large, it may result in extended creation times.

  • It is recommended to keep the Windows C drive below 100G to minimize the initial size. Subsequent expansion must be done manually after creation for Windows systems.

Procedure

Upload ISO Files to DataVolumes

Upload the Windows ISO and virtio-win ISO files directly to the cluster storage using the CDI upload mechanism.

Set up the CDI Upload Proxy

  1. Set up port-forwarding to the CDI upload proxy. This port-forward session will be used for uploading both ISO files.

    kubectl port-forward -n cdi svc/cdi-uploadproxy 8443:443 &
  2. Obtain an authentication token.

    TOKEN=$(kubectl create token default -n default)

Upload the Windows ISO

  1. Create an upload-type DataVolume for the Windows ISO by saving the following YAML to a file named dv-win-iso.yaml. Adjust the storage size based on the actual ISO file size (Windows ISOs are typically 4–6 GB).

    apiVersion: cdi.kubevirt.io/v1beta1
    kind: DataVolume
    metadata:
      name: win-iso-dv
      namespace: default
      annotations:
        cdi.kubevirt.io/storage.bind.immediate.requested: "true"
    spec:
      source:
        upload: {}
      storage:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 8Gi
        storageClassName: vm-cephrbd  # Replace with your actual StorageClass
        volumeMode: Block
  2. Execute the following command to create the DataVolume.

    kubectl apply -f dv-win-iso.yaml
  3. Wait for the DataVolume to enter the UploadReady phase.

    kubectl get dv win-iso-dv -w
    # The PHASE column should show UploadReady
  4. Upload the Windows ISO file using curl. Replace the file path with the actual path to your ISO.

    curl -v --insecure \
      -H "Authorization: Bearer ${TOKEN}" \
      --data-binary @/path/to/en_windows_server_2019_x64_dvd_4cb967d8.iso \
      "https://localhost:8443/v1beta1/upload"
  5. Verify that the DataVolume status changes to Succeeded.

    kubectl get dv win-iso-dv
    # The PHASE column should show Succeeded

Upload the virtio-win ISO

  1. Create an upload-type DataVolume for the virtio-win ISO by saving the following YAML to a file named dv-virtio-iso.yaml.

    apiVersion: cdi.kubevirt.io/v1beta1
    kind: DataVolume
    metadata:
      name: virtio-iso-dv
      namespace: default
      annotations:
        cdi.kubevirt.io/storage.bind.immediate.requested: "true"
    spec:
      source:
        upload: {}
      storage:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
        storageClassName: vm-cephrbd  # Replace with your actual StorageClass
        volumeMode: Block
  2. Execute the following command to create the DataVolume.

    kubectl apply -f dv-virtio-iso.yaml
  3. Wait for the DataVolume to enter the UploadReady phase.

    kubectl get dv virtio-iso-dv -w
    # The PHASE column should show UploadReady
  4. Upload the virtio-win ISO file using curl.

    curl -v --insecure \
      -H "Authorization: Bearer ${TOKEN}" \
      --data-binary @/path/to/virtio-win.iso \
      "https://localhost:8443/v1beta1/upload"
  5. Verify that the DataVolume status changes to Succeeded.

    kubectl get dv virtio-iso-dv
    # The PHASE column should show Succeeded

    Note: The --insecure flag is used to skip self-signed certificate verification. In a production environment, configure proper certificates. Large file uploads may take a long time; ensure network stability.

Create Virtual Machine

  1. Access the Container Platform.

  2. In the left navigation bar, click on Virtualization > Virtual Machines.

  3. Click on Create Virtual Machine.

  4. Fill in the necessary parameters such as Name, Image, etc., in the form page. For detailed parameters and configuration, please refer to Create Virtual Machine.

  5. Switch to YAML.

  6. Replace the configuration under the spec.template.spec.domain.devices field with the following content. The USB tablet input device is required for correct mouse positioning in the VNC console (see kubevirt#2392, kubevirt#3474). Without it, the mouse pointer will be misaligned because VNC uses absolute coordinates while the default PS/2 mouse only supports relative coordinates.

      domain:
        devices:
          inputs:
            - type: tablet
              bus: usb
              name: tablet
          disks:
            - disk:
                bus: virtio
              name: cloudinitdisk
            - bootOrder: 1
              cdrom:
                bus: sata
              name: win-iso
            - cdrom:
                bus: sata
              name: virtio-iso
            - disk:
                bus: sata
              name: rootfs
              bootOrder: 10
  7. Replace the spec.template.spec.volumes field with the following content. Both ISO files are referenced as DataVolumes instead of container images.

          volumes:
            - cloudInitConfigDrive:
                userData: >-
                  #cloud-config
                  disable_root: false
                  ssh_pwauth: true
                  users:
                    - default
                    - name: root
                      lock_passwd: false
                      hashed_passwd: "<hash>"  # Generate with: mkpasswd --method=SHA-512 --rounds=4096
              name: cloudinitdisk
            - dataVolume:
                name: win-iso-dv
              name: win-iso
            - dataVolume:
                name: virtio-iso-dv
              name: virtio-iso
            - dataVolume:
                name: aa-test-rootfs
              name: rootfs
  8. Check the YAML file. The complete YAML after finishing the configuration is as follows.

    apiVersion: kubevirt.io/v1alpha3
    kind: VirtualMachine
    metadata:
      annotations:
        cpaas.io/creator: test@example.io
        cpaas.io/display-name: ""
        cpaas.io/updated-at: 2024-09-01T14:57:55Z
        kubevirt.io/latest-observed-api-version: v1
        kubevirt.io/storage-observed-api-version: v1
      generation: 16
      labels:
        virtualization.cpaas.io/image-name: debian-2120-x86
        virtualization.cpaas.io/image-os-arch: amd64
        virtualization.cpaas.io/image-os-type: debian
        virtualization.cpaas.io/image-supply-by: public
        vm.cpaas.io/name: aa-test
      name: aa-test
      namespace: default
    spec:
      dataVolumeTemplates:
        - metadata:
            creationTimestamp: null
            labels:
              vm.cpaas.io/reclaim-policy: Delete
              vm.cpaas.io/used-by: aa-test
            name: aa-test-rootfs
          spec:
            pvc:
              accessModes:
                - ReadWriteOnce
              resources:
                requests:
                  storage: 100Gi
              storageClassName: vm-cephrbd
              volumeMode: Block
            source:
              blank: {}
      running: true
      template:
        metadata:
          annotations:
            cpaas.io/creator: test@example.io
            cpaas.io/display-name: ""
            cpaas.io/updated-at: 2024-09-01T14:55:44Z
            kubevirt.io/latest-observed-api-version: v1
            kubevirt.io/storage-observed-api-version: v1
          creationTimestamp: null
          labels:
            virtualization.cpaas.io/image-name: debian-2120-x86
            virtualization.cpaas.io/image-os-arch: amd64
            virtualization.cpaas.io/image-os-type: debian
            virtualization.cpaas.io/image-supply-by: public
            vm.cpaas.io/name: aa-test
        spec:
          affinity:
            nodeAffinity: {}
          architecture: amd64
          domain:
            devices:
              inputs:
                - type: tablet
                  bus: usb
                  name: tablet
              disks:
                - disk:
                    bus: virtio
                  name: cloudinitdisk
                - bootOrder: 1
                  cdrom:
                    bus: sata
                  name: win-iso
                - cdrom:
                    bus: sata
                  name: virtio-iso
                - disk:
                    bus: sata
                  name: rootfs
                  bootOrder: 10
              interfaces:
                - bridge: {}
                  name: default
            machine:
              type: q35
            resources:
              limits:
                cpu: "4"
                memory: 8Gi
              requests:
                cpu: "4"
                memory: 8Gi
          networks:
            - name: default
              pod: {}
          nodeSelector:
            kubernetes.io/arch: amd64
            vm.cpaas.io/baremetal: "true"
          volumes:
            - cloudInitConfigDrive:
                userData: >-
                  #cloud-config
                  disable_root: false
                  ssh_pwauth: true
                  users:
                    - default
                    - name: root
                      lock_passwd: false
                      hashed_passwd: "<hash>"  # Generate with: mkpasswd --method=SHA-512 --rounds=4096
              name: cloudinitdisk
            - dataVolume:
                name: win-iso-dv
              name: win-iso
            - dataVolume:
                name: virtio-iso-dv
              name: virtio-iso
            - dataVolume:
                name: aa-test-rootfs
              name: rootfs
  9. Click Create.

  10. Click Actions > VNC Login.

  11. When the prompt press any key boot from CD or DVD appears, press any key to enter the Windows installation program; if you do not see the prompt, click on Send Remote Command in the top left of the page, then select Ctrl-Alt-Delete from the dropdown menu to restart the server.

    Note: If a message appears at the top of the virtual machine details page stating The current virtual machine has configuration changes that require a restart to take effect, please restart, this message can be ignored; no restart is necessary.

Install Windows Operating System

  1. Follow the installation instructions to install the system after entering the installation page.

    Note: During the partition selection step, the bus must be sata for the disk to be correctly recognized. Therefore, you need to select each partition in turn and click Delete to remove all partitions, allowing the system to handle it automatically.

  2. After configuring the administrator account password, click Send Remote Command in the top left of the page, then select Ctrl-Alt-Delete from the dropdown menu.

  3. When prompted The Ctrl+Alt+Delete combination will restart the server, confirm to restart, click OK.

  4. Enter the password to access the Windows system desktop; at this point, the Windows operating system installation is complete.

Install virtio-win-tools

This tool primarily contains the necessary drivers.

  1. Open File Explorer.

  2. Double-click CD Drive(E:) virtio-win-<version>, run the virtio-win-guest-tools directory to enter the installation page, and follow the installation instructions. The <version> part should be based on the actual situation.

  3. After the installation is complete, power off the Windows system.

Export Custom Windows Image

Please refer to Export Virtual Machine Image for the specific operation.

Use Windows Image

  1. Access the Container Platform.

  2. In the left navigation bar, click on Virtualization > Virtual Machines.

  3. Click on Create Virtual Machine.

  4. Fill in the necessary parameters on the form page. For the image, select the exported Windows image. For detailed parameters and configuration, please refer to Create Virtual Machine.

  5. (Optional) If using a newer operating system, such as Windows 11, enable features like clock, UEFI, TPM, etc. Switch to YAML and replace the original YAML file with the following YAML file.

    apiVersion: kubevirt.io/v1
    kind: VirtualMachineInstance
    metadata:
      labels:
        special: vmi-windows
      name: vmi-windows
    spec:
      domain:
        clock:
          timer:
            hpet:
              present: false
            hyperv: {}
            pit:
              tickPolicy: delay
            rtc:
              tickPolicy: catchup
          utc: {}
        cpu:
          cores: 2
        devices:
          inputs:
            - type: tablet
              bus: usb
              name: tablet
          disks:
          - disk:
              bus: sata
            name: pvcdisk
          interfaces:
          - masquerade: {}
            model: e1000
            name: default
          tpm: {}
        features:
          acpi: {}
          apic: {}
          hyperv:
            relaxed: {}
            spinlocks:
              spinlocks: 8191
            vapic: {}
          smm: {}
        firmware:
          bootloader:
            efi:
              secureBoot: true
          uuid: 5d307ca9-b3ef-428c-8861-06e72d69f223
        resources:
          requests:
            memory: 4Gi
      networks:
      - name: default
        pod: {}
      terminationGracePeriodSeconds: 0
      volumes:
      - name: pvcdisk
        persistentVolumeClaim:
          claimName: disk-windows
      - name: winiso
        persistentVolumeClaim:
          claimName: win11cd-pvc
  6. Click Create.

Add Internal Route

By configuring a NodePort type internal route, expose the port for remote desktop connections.

  1. Access the Container Platform.

  2. In the left navigation bar, click on Virtualization > Virtual Machines.

  3. Click on the virtual machine name created with the Windows image in the list to enter the details page.

  4. Click on the Add icon next to Internal Route in the Login Information area.

  5. Configure parameters according to the following instructions.

    ParameterDescription
    TypeSelect NodePort.
    Port
    • Protocol: Select TCP.
    • Service Port: Use 3389.
    • Virtual Machine Port: Use 3389.
    • Service Port Name: Use rdp.
  6. Click OK to return to the details page.

  7. Click on the Internal Route link in the Login Information area.

  8. Save the Virtual IP information in the basic information area and the Host Port information in the port area.

Remote Access

This document discusses using the Windows operating system for remote connection as an example. Other operating systems can use software that supports the RDP protocol for connection.

  1. Open Remote Desktop Connection.

  2. Enter the saved Virtual IP and Host Port from the Add Internal Route step, formatted as Virtual IP:Host Port, for example: 192.1.1.1:3389 .

  3. Click Connect.