QEMU源码全解析 —— virtio（9）

已于 2024-02-13 23:18:31 修改
阅读量526 收藏 9
点赞数 7
分类专栏： QEMU/KVM KVM 文章标签： QEMU KVM virtio
于 2023-12-15 23:28:30 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接： https://blog.csdn.net/phmatthaus/article/details/135026850
版权
QEMU/KVM 同时被 2 个专栏收录
123 篇文章 36 订阅 ¥49.90 ¥99.00
已订阅 8折续费
KVM
88 篇文章 19 订阅
订阅专栏
您已是超级会员，正在免费阅读会员专享内容
查看更多超级会员权益
本文深入剖析QEMU中virtio PCI设备的实现过程，从device_set_realized函数开始，详细解读virtio_pci_dc_realize、pci_qdev_realize以及virtio_pci_realize函数的内部逻辑，包括设备注册、配置空间初始化、BAR数据初始化和VirtIOPCIRegion的设置，逐步揭示virtio设备如何在QEMU中模拟运行。
摘要由CSDN通过智能技术生成

接前一篇文章：

上两回讲解了 virtio balloon相关类所涉及的realize函数以及大致流程，如下表所示：

realize函数 parent_dc_realize函数
DeviceClass virtio_pci_dc_realize
PCIDeviceClass virtio_pci_realize
VirtioPCIClass virtio_balloon_pci_realize pci_qdev_realize

本回对于流程进行深入解析。

综合前文书所讲，当具现化TYPE_VIRTIO_BALLOON的时候，device_set_realized函数中会首先调用DeciceClass->realize即virtio_pci_dc_realize函数。再次贴出该函数源码，在hw/virtio/virtio-pci.c中，如下： 

  1. static void virtio_pci_dc_realize(DeviceState *qdev, Error **errp)
  2. {
  3.     VirtioPCIClass *vpciklass = VIRTIO_PCI_GET_CLASS(qdev);
  4.     VirtIOPCIProxy *proxy = VIRTIO_PCI(qdev);
  5.     PCIDevice *pci_dev = &proxy->pci_dev;
  6.  
  7.     if (!(proxy->flags & VIRTIO_PCI_FLAG_DISABLE_PCIE) &&
  8.         virtio_pci_modern(proxy)) {
  9.         pci_dev->cap_present |= QEMU_PCI_CAP_EXPRESS;
  10.     }
  11.  
  12.     vpciklass->parent_dc_realize(qdev, errp);
  13. }

virtio_pci_dc_realize函数首先判断virtio PCI代理设备是否具有VIRTIO_PCI_FLAG_DISABLE_PCIE特性，该特性使得virtio PCI代理展现出 PCIe 的的接口。

然后调用了vpciklass->parent_dc_realize函数，由前文分析可知，该回调函数是pci_qdev_realize()。再次贴出pci_qdev_realize函数源码，在hw/pci/pci.c中，如下： 

  1. static void pci_qdev_realize(DeviceState *qdev, Error **errp)
  2. {
  3.     PCIDevice *pci_dev = (PCIDevice *)qdev;
  4.     PCIDeviceClass *pc = PCI_DEVICE_GET_CLASS(pci_dev);
  5.     ObjectClass *klass = OBJECT_CLASS(pc);
  6.     Error *local_err = NULL;
  7.     bool is_default_rom;
  8.     uint16_t class_id;
  9.  
  10.     /*
  11.      * capped by systemd (see: udev-builtin-net_id.c)
  12.      * as it's the only known user honor it to avoid users
  13.      * misconfigure QEMU and then wonder why acpi-index doesn't work
  14.      */
  15.     if (pci_dev->acpi_index > ONBOARD_INDEX_MAX) {
  16.         error_setg(errp, "acpi-index should be less or equal to %u",
  17.                    ONBOARD_INDEX_MAX);
  18.         return;
  19.     }
  20.  
  21.     /*
  22.      * make sure that acpi-index is unique across all present PCI devices
  23.      */
  24.     if (pci_dev->acpi_index) {
  25.         GSequence *used_indexes = pci_acpi_index_list();
  26.  
  27.         if (g_sequence_lookup(used_indexes,
  28.                               GINT_TO_POINTER(pci_dev->acpi_index),
  29.                               g_cmp_uint32, NULL)) {
  30.             error_setg(errp, "a PCI device with acpi-index = %" PRIu32
  31.                        " already exist", pci_dev->acpi_index);
  32.             return;
  33.         }
  34.         g_sequence_insert_sorted(used_indexes,
  35.                                  GINT_TO_POINTER(pci_dev->acpi_index),
  36.                                  g_cmp_uint32, NULL);
  37.     }
  38.  
  39.     if (pci_dev->romsize != -1 && !is_power_of_2(pci_dev->romsize)) {
  40.         error_setg(errp, "ROM size %u is not a power of two", pci_dev->romsize);
  41.         return;
  42.     }
  43.  
  44.     /* initialize cap_present for pci_is_express() and pci_config_size(),
  45.      * Note that hybrid PCIs are not set automatically and need to manage
  46.      * QEMU_PCI_CAP_EXPRESS manually */
  47.     if (object_class_dynamic_cast(klass, INTERFACE_PCIE_DEVICE) &&
  48.        !object_class_dynamic_cast(klass, INTERFACE_CONVENTIONAL_PCI_DEVICE)) {
  49.         pci_dev->cap_present |= QEMU_PCI_CAP_EXPRESS;
  50.     }
  51.  
  52.     if (object_class_dynamic_cast(klass, INTERFACE_CXL_DEVICE)) {
  53.         pci_dev->cap_present |= QEMU_PCIE_CAP_CXL;
  54.     }
  55.  
  56.     pci_dev = do_pci_register_device(pci_dev,
  57.                                      object_get_typename(OBJECT(qdev)),
  58.                                      pci_dev->devfn, errp);
  59.     if (pci_dev == NULL)
  60.         return;
  61.  
  62.     if (pc->realize) {
  63.         pc->realize(pci_dev, &local_err);
  64.         if (local_err) {
  65.             error_propagate(errp, local_err);
  66.             do_pci_unregister_device(pci_dev);
  67.             return;
  68.         }
  69.     }
  70.  
  71.     /*
  72.      * A PCIe Downstream Port that do not have ARI Forwarding enabled must
  73.      * associate only Device 0 with the device attached to the bus
  74.      * representing the Link from the Port (PCIe base spec rev 4.0 ver 0.3,
  75.      * sec 7.3.1).
  76.      * With ARI, PCI_SLOT() can return non-zero value as the traditional
  77.      * 5-bit Device Number and 3-bit Function Number fields in its associated
  78.      * Routing IDs, Requester IDs and Completer IDs are interpreted as a
  79.      * single 8-bit Function Number. Hence, ignore ARI capable devices.
  80.      */
  81.     if (pci_is_express(pci_dev) &&
  82.         !pcie_find_capability(pci_dev, PCI_EXT_CAP_ID_ARI) &&
  83.         pcie_has_upstream_port(pci_dev) &&
  84.         PCI_SLOT(pci_dev->devfn)) {
  85.         warn_report("PCI: slot %d is not valid for %s,"
  86.                     " parent device only allows plugging into slot 0.",
  87.                     PCI_SLOT(pci_dev->devfn), pci_dev->name);
  88.     }
  89.  
  90.     if (pci_dev->failover_pair_id) {
  91.         if (!pci_bus_is_express(pci_get_bus(pci_dev))) {
  92.             error_setg(errp, "failover primary device must be on "
  93.                              "PCIExpress bus");
  94.             pci_qdev_unrealize(DEVICE(pci_dev));
  95.             return;
  96.         }
  97.         class_id = pci_get_word(pci_dev->config + PCI_CLASS_DEVICE);
  98.         if (class_id != PCI_CLASS_NETWORK_ETHERNET) {
  99.             error_setg(errp, "failover primary device is not an "
  100.                              "Ethernet device");
  101.             pci_qdev_unrealize(DEVICE(pci_dev));
  102.             return;
  103.         }
  104.         if ((pci_dev->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)
  105.             || (PCI_FUNC(pci_dev->devfn) != 0)) {
  106.             error_setg(errp, "failover: primary device must be in its own "
  107.                               "PCI slot");
  108.             pci_qdev_unrealize(DEVICE(pci_dev));
  109.             return;
  110.         }
  111.         qdev->allow_unplug_during_migration = true;
  112.     }
  113.  
  114.     /* rom loading */
  115.     is_default_rom = false;
  116.     if (pci_dev->romfile == NULL && pc->romfile != NULL) {
  117.         pci_dev->romfile = g_strdup(pc->romfile);
  118.         is_default_rom = true;
  119.     }
  120.  
  121.     pci_add_option_rom(pci_dev, is_default_rom, &local_err);
  122.     if (local_err) {
  123.         error_propagate(errp, local_err);
  124.         pci_qdev_unrealize(DEVICE(pci_dev));
  125.         return;
  126.     }
  127.  
  128.     pci_set_power(pci_dev, true);
  129.  
  130.     pci_dev->msi_trigger = pci_msi_trigger;
  131. }

pci_qdev_realize函数会将virtioPCI代理设备注册到PCI总线上，如前文所示：

并调用PCIDeviceClass->realize函数指针所指向的函数，也就是virtio_pci_realize函数。代码片段如下： 

  1.     pci_dev = do_pci_register_device(pci_dev,
  2.                                      object_get_typename(OBJECT(qdev)),
  3.                                      pci_dev->devfn, errp);
  4.     if (pci_dev == NULL)
  5.         return;
  6.  
  7.     if (pc->realize) {
  8.         pc->realize(pci_dev, &local_err);
  9.         if (local_err) {
  10.             error_propagate(errp, local_err);
  11.             do_pci_unregister_device(pci_dev);
  12.             return;
  13.         }
  14.     }

关于pci_qdev_realize函数的详细讲解可以参阅笔者的博文 QEMU源码全解析 —— PCI设备模拟（2） 。这里为了便于理解，将其中部分内容再次贴出：

pci_qdev_realize函数主要包括三个方面的工作：

（1）首先调用do_pci_register_device函数进行注册。

do_pci_register_device函数完成设备及其对应PCI总线上的一些初始化工作。具体包括：

1）如果指定的devfn为-1，表示由总线自己选择插槽，得到插槽之后保存在PCIDevice的devfn（即pci_dev->devfn）中；如果在设备命令行中指定了addr，则addr会作为设备的devfn。

2）接下来设置PCIDevice结构体中的各个域，包括调用pci_init_bus_master函数初始化PCIDevice中的 Address 成员bus_master_as及其对应的MR。

3）之后，调用pci_config_alloc函数分配PCI设备的配置空间，cmask用来检测相关的能力，wmask用来控制读写，w1cmask用来实现RW1C。由此完成一些初始化的设置，如vendor_id等。

4）然后是设置设备的config_read和config_write函数。如果相关的子类自己没有设置，那么就使用默认的pci_default_read/write_config函数。

5）最后，将该device复制到bus->devices数组中。

（2）其次，pci_qdev_realize函数调用PCI设备所属的class的realize函数，即pc->realize函数，也就是virtio_pci_realize函数 。

（3）最后，调用pci_add_option_rom函数加载PCI设备的ROM。

virtio_pci_realize函数在hw/virtio/virtio-pci.c中，代码如下： 

  1. static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp)
  2. {
  3.     VirtIOPCIProxy *proxy = VIRTIO_PCI(pci_dev);
  4.     VirtioPCIClass *k = VIRTIO_PCI_GET_CLASS(pci_dev);
  5.     bool pcie_port = pci_bus_is_express(pci_get_bus(pci_dev)) &&
  6.                      !pci_bus_is_root(pci_get_bus(pci_dev));
  7.  
  8.     if (kvm_enabled() && !kvm_has_many_ioeventfds()) {
  9.         proxy->flags &= ~VIRTIO_PCI_FLAG_USE_IOEVENTFD;
  10.     }
  11.  
  12.     /* fd-based ioevents can't be synchronized in record/replay */
  13.     if (replay_mode != REPLAY_MODE_NONE) {
  14.         proxy->flags &= ~VIRTIO_PCI_FLAG_USE_IOEVENTFD;
  15.     }
  16.  
  17.     /*
  18.      * virtio pci bar layout used by default.
  19.      * subclasses can re-arrange things if needed.
  20.      *
  21.      *   region 0   --  virtio legacy io bar
  22.      *   region 1   --  msi-x bar
  23.      *   region 2   --  virtio modern io bar (off by default)
  24.      *   region 4+5 --  virtio modern memory (64bit) bar
  25.      *
  26.      */
  27.     proxy->legacy_io_bar_idx  = 0;
  28.     proxy->msix_bar_idx       = 1;
  29.     proxy->modern_io_bar_idx  = 2;
  30.     proxy->modern_mem_bar_idx = 4;
  31.  
  32.     proxy->common.offset = 0x0;
  33.     proxy->common.size = 0x1000;
  34.     proxy->common.type = VIRTIO_PCI_CAP_COMMON_CFG;
  35.  
  36.     proxy->isr.offset = 0x1000;
  37.     proxy->isr.size = 0x1000;
  38.     proxy->isr.type = VIRTIO_PCI_CAP_ISR_CFG;
  39.  
  40.     proxy->device.offset = 0x2000;
  41.     proxy->device.size = 0x1000;
  42.     proxy->device.type = VIRTIO_PCI_CAP_DEVICE_CFG;
  43.  
  44.     proxy->notify.offset = 0x3000;
  45.     proxy->notify.size = virtio_pci_queue_mem_mult(proxy) * VIRTIO_QUEUE_MAX;
  46.     proxy->notify.type = VIRTIO_PCI_CAP_NOTIFY_CFG;
  47.  
  48.     proxy->notify_pio.offset = 0x0;
  49.     proxy->notify_pio.size = 0x4;
  50.     proxy->notify_pio.type = VIRTIO_PCI_CAP_NOTIFY_CFG;
  51.  
  52.     /* subclasses can enforce modern, so do this unconditionally */
  53.     memory_region_init(&proxy->modern_bar, OBJECT(proxy), "virtio-pci",
  54.                        /* PCI BAR regions must be powers of 2 */
  55.                        pow2ceil(proxy->notify.offset + proxy->notify.size));
  56.  
  57.     if (proxy->disable_legacy == ON_OFF_AUTO_AUTO) {
  58.         proxy->disable_legacy = pcie_port ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
  59.     }
  60.  
  61.     if (!virtio_pci_modern(proxy) && !virtio_pci_legacy(proxy)) {
  62.         error_setg(errp, "device cannot work as neither modern nor legacy mode"
  63.                    " is enabled");
  64.         error_append_hint(errp, "Set either disable-modern or disable-legacy"
  65.                           " to off\n");
  66.         return;
  67.     }
  68.  
  69.     if (pcie_port && pci_is_express(pci_dev)) {
  70.         int pos;
  71.         uint16_t last_pcie_cap_offset = PCI_CONFIG_SPACE_SIZE;
  72.  
  73.         pos = pcie_endpoint_cap_init(pci_dev, 0);
  74.         assert(pos > 0);
  75.  
  76.         pos = pci_add_capability(pci_dev, PCI_CAP_ID_PM, 0,
  77.                                  PCI_PM_SIZEOF, errp);
  78.         if (pos < 0) {
  79.             return;
  80.         }
  81.  
  82.         pci_dev->exp.pm_cap = pos;
  83.  
  84.         /*
  85.          * Indicates that this function complies with revision 1.2 of the
  86.          * PCI Power Management Interface Specification.
  87.          */
  88.         pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3);
  89.  
  90.         if (proxy->flags & VIRTIO_PCI_FLAG_AER) {
  91.             pcie_aer_init(pci_dev, PCI_ERR_VER, last_pcie_cap_offset,
  92.                           PCI_ERR_SIZEOF, NULL);
  93.             last_pcie_cap_offset += PCI_ERR_SIZEOF;
  94.         }
  95.  
  96.         if (proxy->flags & VIRTIO_PCI_FLAG_INIT_DEVERR) {
  97.             /* Init error enabling flags */
  98.             pcie_cap_deverr_init(pci_dev);
  99.         }
  100.  
  101.         if (proxy->flags & VIRTIO_PCI_FLAG_INIT_LNKCTL) {
  102.             /* Init Link Control Register */
  103.             pcie_cap_lnkctl_init(pci_dev);
  104.         }
  105.  
  106.         if (proxy->flags & VIRTIO_PCI_FLAG_INIT_PM) {
  107.             /* Init Power Management Control Register */
  108.             pci_set_word(pci_dev->wmask + pos + PCI_PM_CTRL,
  109.                          PCI_PM_CTRL_STATE_MASK);
  110.         }
  111.  
  112.         if (proxy->flags & VIRTIO_PCI_FLAG_ATS) {
  113.             pcie_ats_init(pci_dev, last_pcie_cap_offset,
  114.                           proxy->flags & VIRTIO_PCI_FLAG_ATS_PAGE_ALIGNED);
  115.             last_pcie_cap_offset += PCI_EXT_CAP_ATS_SIZEOF;
  116.         }
  117.  
  118.         if (proxy->flags & VIRTIO_PCI_FLAG_INIT_FLR) {
  119.             /* Set Function Level Reset capability bit */
  120.             pcie_cap_flr_init(pci_dev);
  121.         }
  122.     } else {
  123.         /*
  124.          * make future invocations of pci_is_express() return false
  125.          * and pci_config_size() return PCI_CONFIG_SPACE_SIZE.
  126.          */
  127.         pci_dev->cap_present &= ~QEMU_PCI_CAP_EXPRESS;
  128.     }
  129.  
  130.     virtio_pci_bus_new(&proxy->bus, sizeof(proxy->bus), proxy);
  131.     if (k->realize) {
  132.         k->realize(proxy, errp);
  133.     }
  134. }

（1）virtio_pci_realize函数首先得到virtioPCIProxy设备，也就是VirtIOPCIProxy结构的对象。代码片段如下： 

    VirtIOPCIProxy *proxy = VIRTIO_PCI(pci_dev);

（2）然后，virtio_pci_realize函数初始化VirtIOPCIProxy设备的多个BAR数据，设置了这些BAR的索引号。代码片段如下： 

  1.     /*
  2.      * virtio pci bar layout used by default.
  3.      * subclasses can re-arrange things if needed.
  4.      *
  5.      *   region 0   --  virtio legacy io bar
  6.      *   region 1   --  msi-x bar
  7.      *   region 2   --  virtio modern io bar (off by default)
  8.      *   region 4+5 --  virtio modern memory (64bit) bar
  9.      *
  10.      */
  11.     proxy->legacy_io_bar_idx  = 0;
  12.     proxy->msix_bar_idx       = 1;
  13.     proxy->modern_io_bar_idx  = 2;
  14.     proxy->modern_mem_bar_idx = 4;

其中：

legacy I/O地址为0；

msi-x地址为1；

modern IO地址为2；

modern MMIO地址为4。

这里的legacy和modern指的是不同的virtio版本。

（3）virtio_pci_realize函数还初始化了多个VirtIOPCIRegion（VirtIOPCIRegion用来表示virtio设备的配置空间信息），如VirtIOPCIProxy的common、isr、device、notify成员。代码片段如下： 

  1.     proxy->common.offset = 0x0;
  2.     proxy->common.size = 0x1000;
  3.     proxy->common.type = VIRTIO_PCI_CAP_COMMON_CFG;
  4.  
  5.     proxy->isr.offset = 0x1000;
  6.     proxy->isr.size = 0x1000;
  7.     proxy->isr.type = VIRTIO_PCI_CAP_ISR_CFG;
  8.  
  9.     proxy->device.offset = 0x2000;
  10.     proxy->device.size = 0x1000;
  11.     proxy->device.type = VIRTIO_PCI_CAP_DEVICE_CFG;
  12.  
  13.     proxy->notify.offset = 0x3000;
  14.     proxy->notify.size = virtio_pci_queue_mem_mult(proxy) * VIRTIO_QUEUE_MAX;
  15.     proxy->notify.type = VIRTIO_PCI_CAP_NOTIFY_CFG;
  16.  
  17.     proxy->notify_pio.offset = 0x0;
  18.     proxy->notify_pio.size = 0x4;
  19.     proxy->notify_pio.type = VIRTIO_PCI_CAP_NOTIFY_CFG;

VirtIOPCIRegion保存了VirtIOPCIProxy设备modern MMIO的相关信息。如VirtIOProxy的modern MMIO中，最开始的区域是common区域，其大小为0x1000，范围是[0x0, 0x1000)；接着是isr区域，大小也是0x1000，范围是[0x1000, 0x2000)；然后是device区域，其大小仍然是0x1000，范围是[0x2000,0x3000)；最后是notify区域，其大小是virtio_pci_queue_mem_mult(proxy) * VIRTIO_QUEUE_MAX，范围是[0x3000, 0x3000+virtio_pci_queue_mem_mult(proxy) * VIRTIO_QUEUE_MAX)。

（4）VirtIOPCIProxy的modern MMIO对应的MemoryRegion存放在VirtIOPCIProxy的modern_bar成员中；它还有一个MemoryRegion存放在modern_cfg成员中（这部分老版本代码有，新版本代码已经没有了）。代码片段如下： 

  1.     /* subclasses can enforce modern, so do this unconditionally */
  2.     memory_region_init(&proxy->modern_bar, OBJECT(proxy), "virtio-pci",
  3.                        /* PCI BAR regions must be powers of 2 */
  4.                        pow2ceil(proxy->notify.offset + proxy->notify.size));
  5.  
  6.     if (proxy->disable_legacy == ON_OFF_AUTO_AUTO) {
  7.         proxy->disable_legacy = pcie_port ? ON_OFF_AUTO_ON : ON_OFF_AUTO_OFF;
  8.     }
  9.  
  10.     if (!virtio_pci_modern(proxy) && !virtio_pci_legacy(proxy)) {
  11.         error_setg(errp, "device cannot work as neither modern nor legacy mode"
  12.                    " is enabled");
  13.         error_append_hint(errp, "Set either disable-modern or disable-legacy"
  14.                           " to off\n");
  15.         return;
  16.     }

（5）virtio_pci_realize函数会调用virtio_pci_bus_new函数创建virtio-bus，挂载到当前的virtio PCI代理设备下面。代码片段如下： 

    virtio_pci_bus_new(&proxy->bus, sizeof(proxy->bus), proxy);

（6）virtio_pci_realize函数在最后调用了k->realize函数指针所指向的函数。由于k是VirtioPCIClass类型，因此根据前表

realize函数 parent_dc_realize函数
DeviceClass virtio_pci_dc_realize
PCIDeviceClass virtio_pci_realize
VirtioPCIClass virtio_balloon_pci_realize pci_qdev_realize

因此k->realize函数指针实际指向了virtio_balloon_pci_realize函数。

对于virtio_balloon_pci_realize函数内容的讲解，请看下回。

举报

选择你想要举报的内容（必选）
  • 内容涉黄
  • 政治相关
  • 内容抄袭
  • 涉嫌广告
  • 内容侵权
  • 侮辱谩骂
  • 样式问题
  • 其他

取消

确定