Creating Linux Images Based on ISO Using KubeVirt

This document describes a virtual machine solution implemented based on the open-source component KubeVirt. It utilizes KubeVirt virtualization technology to create a Linux operating system image from an ISO image file. The ISO is uploaded to the cluster via CDI DataVolume, eliminating the need to build and push container images to a registry.

Prerequisites

  • All components in the cluster are functioning properly.

  • A Linux image should be prepared in advance. This document uses the Ubuntu operating system as an example.

  • The kubectl command-line tool is installed and configured to access the cluster.

  • A StorageClass that supports ReadWriteOnce (RWO) access mode is available in the cluster.

Constraints and Limitations

  • When starting KubeVirt, the file system size of the custom image will affect the speed of writing the image to the PVC disk. If the file system is too large, it may result in a prolonged creation time.

  • It is recommended to keep the Linux root partition size below 100G to minimize the initial size. After configuring cloud-init, allocate larger storage for the root partition when creating the virtual machine, and the system will automatically expand it.

Procedure

Upload the Linux ISO to a DataVolume

Upload the ISO file directly to the cluster storage using the CDI upload mechanism, without building a container image.

  1. Create an upload-type DataVolume by saving the following YAML to a file named dv-iso-upload.yaml. Adjust the storage size based on the actual ISO file size.

    apiVersion: cdi.kubevirt.io/v1beta1
    kind: DataVolume
    metadata:
      name: iso-upload-dv
      namespace: default
      annotations:
        cdi.kubevirt.io/storage.bind.immediate.requested: "true"
    spec:
      source:
        upload: {}
      storage:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 8Gi
        storageClassName: vm-cephrbd  # Replace with your actual StorageClass
        volumeMode: Block
  2. Execute the following command to create the DataVolume.

    kubectl apply -f dv-iso-upload.yaml
  3. Wait for the DataVolume to enter the UploadReady phase.

    kubectl get dv iso-upload-dv -w
    # The PHASE column should show UploadReady
  4. Set up port-forwarding to the CDI upload proxy.

    kubectl port-forward -n cdi svc/cdi-uploadproxy 8443:443 &
  5. Obtain an authentication token.

    TOKEN=$(kubectl create token default -n default)
  6. Upload the ISO file using curl. Replace the file path with the actual path to your ISO.

    curl -v --insecure \
      -H "Authorization: Bearer ${TOKEN}" \
      --data-binary @/path/to/ubuntu-24.04-live-server-amd64.iso \
      "https://localhost:8443/v1beta1/upload"

    Note: The --insecure flag is used to skip self-signed certificate verification. In a production environment, configure proper certificates. Large file uploads may take a long time; ensure network stability.

  7. Verify that the DataVolume status changes to Succeeded.

    kubectl get dv iso-upload-dv
    # The PHASE column should show Succeeded

Create Virtual Machine

  1. Enter the Container Platform.

  2. Click Virtualization > Virtual Machines in the left navigation bar.

  3. Click Create Virtual Machine.

  4. Fill in the parameters on the form page as follows. For specific parameters and configurations, please refer to Create Virtual Machine.

    ParameterDescription
    Select ImageChoose the template image for the virtual machine.
    IP AddressKeep default, which will be obtained via DHCP.
    Network ModeUse NAT mode; do not use bridged mode here.
  5. Switch to YAML.

  6. Replace the configuration under the spec.template.spec.domain.devices.disks field with the following content. The ISO DataVolume is mounted as a CDROM with the highest boot priority, and the rootfs disk is used as the installation target.

          domain:
            devices:
              disks:
                - bootOrder: 1
                  cdrom:
                    bus: sata
                  name: iso-disk
                - disk:
                    bus: virtio
                  name: cloudinitdisk
                - disk:
                    bus: virtio
                  name: rootfs
                  bootOrder: 10
  7. Replace the spec.template.spec.volumes field with the following content. The ISO is referenced as a DataVolume instead of a container image.

          volumes:
            - dataVolume:
                name: iso-upload-dv
              name: iso-disk
            - cloudInitConfigDrive:
                userData: |-
                  #cloud-config
                  disable_root: false
                  ssh_pwauth: false
                  users:
                    - default
                    - name: root
                      lock_passwd: false
                      hashed_passwd: "<hash>"  # Generate with: mkpasswd --method=SHA-512 --rounds=4096
              name: cloudinitdisk
            - dataVolume:
                name: aa-rootfs
              name: rootfs
  8. Review the YAML file; the complete YAML configuration after completion is as follows.

    apiVersion: kubevirt.io/v1alpha3
    kind: VirtualMachine
    metadata:
      annotations:
        kubevirt.io/latest-observed-api-version: v1
        kubevirt.io/storage-observed-api-version: v1
      labels:
        virtualization.cpaas.io/image-name: debian-2120-x86
        virtualization.cpaas.io/image-os-arch: amd64
        virtualization.cpaas.io/image-os-type: debian
        virtualization.cpaas.io/image-supply-by: public
        vm.cpaas.io/name: aa
      name: aa
    spec:
      dataVolumeTemplates:
        - metadata:
            creationTimestamp: null
            labels:
              vm.cpaas.io/reclaim-policy: Delete
              vm.cpaas.io/used-by: aa
            name: aa-rootfs
          spec:
            pvc:
              accessModes:
                - ReadWriteOnce
              resources:
                requests:
                  storage: 100Gi
              storageClassName: vm-cephrbd
              volumeMode: Block
            source:
              blank: {}
      running: true
      template:
        metadata:
          annotations:
            cpaas.io/creator: test@example.io
            cpaas.io/display-name: ""
            cpaas.io/updated-at: 2024-09-09T03:49:08Z
            kubevirt.io/latest-observed-api-version: v1
            kubevirt.io/storage-observed-api-version: v1
          creationTimestamp: null
          labels:
            virtualization.cpaas.io/image-name: debian-2120-x86
            virtualization.cpaas.io/image-os-arch: amd64
            virtualization.cpaas.io/image-os-type: debian
            virtualization.cpaas.io/image-supply-by: public
            vm.cpaas.io/name: aa
        spec:
          accessCredentials:
            - sshPublicKey:
                propagationMethod:
                  qemuGuestAgent:
                    users:
                      - root
                source:
                  secret:
                    secretName: test-xeon
          affinity:
            nodeAffinity: {}
          architecture: amd64
          domain:
            devices:
              disks:
                - bootOrder: 1
                  cdrom:
                    bus: sata
                  name: iso-disk
                - disk:
                    bus: virtio
                  name: cloudinitdisk
                - disk:
                    bus: virtio
                  name: rootfs
                  bootOrder: 10
              interfaces:
                - bridge: {}
                  name: default
            machine:
              type: q35
            resources:
              limits:
                cpu: "1"
                memory: 2Gi
              requests:
                cpu: "1"
                memory: 2Gi
          networks:
            - name: default
              pod: {}
          nodeSelector:
            kubernetes.io/arch: amd64
            vm.cpaas.io/baremetal: "true"
          volumes:
            - dataVolume:
                name: iso-upload-dv
              name: iso-disk
            - cloudInitConfigDrive:
                userData: |-
                  #cloud-config
                  disable_root: false
                  ssh_pwauth: false
                  users:
                    - default
                    - name: root
                      lock_passwd: false
                      hashed_passwd: "<hash>"  # Generate with: mkpasswd --method=SHA-512 --rounds=4096
              name: cloudinitdisk
            - dataVolume:
                name: aa-rootfs
              name: rootfs
  9. Click Create.

  10. Click Actions > VNC Login.

  11. When prompted with press any key boot from CD or DVD, press any key to enter the installation program; if you do not see the prompt, click Send Remote Command in the upper left corner of the page, and then click Ctrl-Alt-Delete from the dropdown menu to reboot the server.

    Note: If a message appears at the top of the virtual machine detail page stating Current virtual machine has configuration changes that require a restart to take effect. Please restart., you can ignore this message; a restart is not necessary.

Install Linux Operating System

  1. After entering the installation page, follow the installation guide to proceed. This document gives an example of installing the Ubuntu operating system; the configuration items during the installation process of different operating systems are generally similar, and thus will not be elaborated further. Some configuration items are explained below.

    ConfigurationDescription
    Installation TypeIt is recommended to use a minimal installation to minimize the image size.
    Storage ConfigurationChoose custom storage. Format the disk to ext4 or xfs format and mount it to the root partition (/).
    Note: Do not use LVM for disk partitioning (Create volume group (LVM)).
    SSH ConfigurationChoose to install the OpenSSH tools for SSH access.
  2. Wait for the installation to complete.

Modify YAML File

  1. Enter the Container Platform.

  2. In the left navigation bar, click Virtualization > Virtual Machines.

  3. Click on the Virtual Machine Name in the list to enter the details page.

  4. Click Stop.

  5. Click Actions > Update in the upper right corner.

  6. Switch to YAML.

  7. Confirm that the disk named rootfs under spec.template.spec.domain.devices.disks has a bootOrder of 1. If it is not 1, modify it to 1.

  8. Delete the relevant content for the ISO disk named iso-disk under spec.template.spec.domain.devices.disks; the specific content to delete is as follows.

                - bootOrder: 1
                  cdrom:
                    bus: sata
                  name: iso-disk
  9. Delete the relevant content for the volume named iso-disk under spec.template.spec.volumes; the specific content to delete is as follows.

            - dataVolume:
                name: iso-upload-dv
              name: iso-disk
  10. Click Update.

  11. Click Start.

Install Required Software and Modify Configuration

Note: The following commands and configuration files may vary slightly between different operating systems; please adjust according to your actual environment.

  1. Enter your username and password to log in to the operating system.

  2. Switch to root user privileges.

  3. Install the software packages.

    • For CentOS series, execute the command:

      yum install cloud-utils cloud-init qemu-guest-agent vim
    • For Debian series, execute the command:

      apt install cloud-init cloud-guest-utils qemu-guest-agent vim
  4. Edit the SSHD configuration file.

    1. Execute the following command to edit the sshd_config file.

      vim /etc/ssh/sshd_config
    2. Add the following configurations.

      PermitRootLogin yes  # Allow the root user to log in with a password
      PubkeyAuthentication yes  # Allow key-based login
    3. Save the modified configuration.

  5. Execute the following command to delete the default password for the root user.

    passwd -d root
  6. Modify the source address file.

    1. Execute the following command to modify the system's source address file and change the address to a suitable mirror site address.

      vim /etc/apt/sources.list.d/ubuntu.sources
    2. Save the configuration after modifications.

  7. Modify the cloud-init configuration to automatically expand the root directory.

    1. Execute the following command to edit the cloud.cfg configuration file.

      vim /etc/cloud/cloud.cfg
    2. Add the following configuration content.

      runcmd:
        - [growpart, /dev/vda, 1]  # The growpart command is used to extend the partition on the disk, which will extend the /dev/vda1 partition.
        - [xfs_growfs, /dev/vda1]  # The xfs_growfs command is used to extend the XFS file system to occupy all available space on the partition. /dev/vda1 is the partition where the file system to be extended is located. After extending the partition, using xfs_growfs ensures that the file system itself is also expanded to the new partition size.
    3. Save the configuration after modifications.

  8. After completing the configuration, shut down the operating system.

Export and Use the Custom Linux Image

For specific operations, please refer to Export Virtual Machine Image.