Ovirt中的云主机监控

Ovirt中云主机监控

1. PollVmStatsRefresher.java Vm刷新调度器

   • poll() 调度执行方法

     @OnTimerMethodAnnotation("poll")
     public void poll() {
         if (isMonitoringNeeded(vdsManager.getStatus())) {
             //负责获取虚拟机列表，并将其存储在VdsManager上进行分析
                  VmsListFetcher fetcher = new VmsStatisticsFetcher(vdsManager);
                 long fetchTime = System.nanoTime();
                 //fetcher.fetch()  调用vdsm接口，获取AllVmStats信息
                 if (fetcher.fetch()) {
                     //分析监控到的主机的数据变化并做出处理。相关的更改将被持久化，状态转换和内部命令将相应地发生
                     getVmsMonitoring().perform(fetcher.getChangedVms(), fetchTime, vdsManager, true);
                     Stream<VdsmVm> vdsmVmsToMonitor = filterVmsToDevicesMonitoring(fetcher.getChangedVms());
                     processDevices(vdsmVmsToMonitor,fetchTime);
                 } else {
                     log.info("Failed to fetch vms info for host '{}' - skipping VMs monitoring.",vdsManager.getVdsName());
                 }
             }
         }   

   • startMonitoring() 启动调度 10-15s

     public void startMonitoring() {
         vmsMonitoringJob =
         schedulerService.scheduleWithFixedDelay(
                         this::poll,
                         VMS_REFRESH_RATE * NUMBER_VMS_REFRESHES_BEFORE_SAVE,
                         VMS_REFRESH_RATE * NUMBER_VMS_REFRESHES_BEFORE_SAVE,
                         TimeUnit.MILLISECONDS);
     }

2. VmsStatisticsFetcher.java 由PollVmStatsRefresher.poll()调用

       VmsListFetcher fetcher = new VmsStatisticsFetcher(vdsManager);
       fetcher.fetch()  //该方法会执行vdsm接口

   • fetch() 由VmsStatisticsFetcher继承VmsListFetcher

         public boolean fetch() {
             VDSReturnValue pollReturnValue = poll();
             if (pollReturnValue.getSucceeded()) {
                 vdsmVms = (Map<Guid, VdsmVm>) pollReturnValue.getReturnValue();
                 //该方法会对获取道德信息进行过滤，过滤处修改后的vms信息，并且会将VmDynamic信息存储到库中
                 onFetchVms();
                 return true;
             } else {
                 onError();
                 return false;
             }
         }

   • poll() 调用vdsm接口GetAllVmStats poll()会被VmsStatisticsFetcher重写

         @Override
         protected VDSReturnValue poll() {
             return getResourceManager().runVdsCommand(
                     VDSCommandType.GetAllVmStats,
                     new VdsIdVDSCommandParametersBase(vdsManager.getVdsId()));
         }

     • onFetchVms()

       protected void onFetchVms() {
          //获取当前库中的vms信息
           dbVms = getVmDynamicDao().getAllRunningForVds(vdsManager.getVdsId()).stream()
                       .collect(Collectors.toMap(VmDynamic::getId, Function.identity()));
           changedVms = new ArrayList<>();
           //过滤和比较dbVms、vdsmVms，如果vm状态发生改变，则重新调用vdsm接口GetVmStats，获取最新数据到changeVms中
          filterVms();
           //过滤没有运行的云主机
          gatherNonRunningVms(dbVms);
           //保存最新的vms列表
           saveLastVmsList();
       }

3. VmsMonitoring 调用手头的所有Vm分析器，遍历他们的报告并采取行动-触发VDSM命令(destroy,runrerun,migrate)，报告完成的操作，移交迁移和保存到db

   • perform() 由PollVmStatsRefresher.poll()调用

        public void perform(
            List<Pair<VmDynamic, VdsmVm>> monitoredVms,
            long fetchTime,
            VdsManager vdsManager,
            boolean updateStatistics) {
            if (monitoredVms.isEmpty()) {
                return;
            }
     
            List<VmAnalyzer> vmAnalyzers = Collections.emptyList();
            try {
                vmAnalyzers = analyzeVms(monitoredVms, fetchTime, vdsManager, updateStatistics);
                // It is important to add the unmanaged VMs before flushing the dynamic data into the database
                addUnmanagedVms(vmAnalyzers, vdsManager.getVdsId());
                flush(vmAnalyzers);
                postFlush(vmAnalyzers, vdsManager, fetchTime);
                vdsManager.vmsMonitoringInitFinished();
            } catch (RuntimeException ex) {
                log.error("Failed during vms monitoring on host {} error is: {}", vdsManager.getVdsName(), ex);
                log.error("Exception:", ex);
            } finally {
                unlockVms(vmAnalyzers);
            }
     
        }

     • analyzeVms() 分析虚拟机，获取分析结果

       /**
       * Analyze the VM data pair
       * Skip analysis on VMs which cannot be locked
       * note: metrics calculation like memCommited and vmsCoresCount should be calculated *before*
       *   this filtering.
       * @return The analyzers which hold all the data per VM
       */
       private List<VmAnalyzer> analyzeVms(
           List<Pair<VmDynamic, VdsmVm>> monitoredVms,
           long fetchTime,
           VdsManager vdsManager,
           boolean updateStatistics) {
       VmAnalyzerFactory vmAnalyzerFactory = getVmAnalyzerFactory(vdsManager, updateStatistics);
       List<VmAnalyzer> vmAnalyzers = new ArrayList<>(monitoredVms.size());
       monitoredVms.forEach(vm -> {
           // TODO filter out migratingTo VMs if no action is taken on them
           if (shouldAnalyzeVm(vm, fetchTime, vdsManager.getVdsId())) {
               try {
                   VmAnalyzer vmAnalyzer = vmAnalyzerFactory.getVmAnalyzer(vm);
                   //该方法根据不同云主机状态去封装对象
                   vmAnalyzer.analyze();
                   vmAnalyzers.add(vmAnalyzer);
               } catch (RuntimeException ex) {
                   Guid vmId = getVmId(vm.getFirst(), vm.getSecond());
                   VmManager vmManager = getVmManager(vmId);
                   vmManager.unlock();
       
                   log.error("Failed during monitoring vm: {} , error is: {}", vmId, ex);
                   log.error("Exception:", ex);
               }
           }
       });
       vmAnalyzers.sort(Comparator.comparing(VmAnalyzer::getVmId));
       return vmAnalyzers;
       }

   • flush(vmAnalyzers) 保存vm数据

        private void flush(List<VmAnalyzer> vmAnalyzers) {
            saveVmDynamic(vmAnalyzers);
            saveVmStatistics(vmAnalyzers);
            saveVmInterfaceStatistics(vmAnalyzers);
            saveVmDiskImageStatistics(vmAnalyzers);
            saveVmGuestAgentNetworkDevices(vmAnalyzers);
        }

   总结

   1. 在云主机中，监控信息通过vm_interface_view查询出来的，vm_interface_view 查询表vm_interface_statistics中的数据表。

   2. ovirt对云主机有一个调度任务，去获取vds主机上实际云主机信息，来同步本地云主机。大致过程为：

      1. 创建调度方法，并启动调度。
      2. 在调度方法中，ovirt判断vds的状态是否需要监控
      3. ovirt会先调用vdsm接口GetAllVmStats获取全部的vmsStatic信息，并与当前库中的dbVms信息作对比，主要比较dbvm是否为空，以及双方状态是否一致
      4. 比较之后会重新调用vdsm接口GetVmStats获取需要修改的vms信息
      5. ovirt会根据修改的vms信息，分析其变化并做出反应。相关的更改将被持久化，状态转换和内部命令将相应地发生

   3. 云主机网络波动显示，也是由监控调度同步而来。VmAnalyzer.updateInterfaceStatistics()

      1. 从vdsm获取值为integer类型，在ovirt会进行最大值判断，当vdsm返回时大于100，按100取。

      2. 网络波动最多显示数量为40

         statistics.setUsageNetworkPercent(min(statistics.getUsageNetworkPercent(), 100));
           Integer usageHistoryLimit = Config.getValue(ConfigValues.UsageHistoryLimit);
         statistics.addNetworkUsageHistory(statistics.getUsageNetworkPercent(), usageHistoryLimit);

   4. 云主机监控信息（rx、tx速率）由监控调度同步而来。
      ``VmAnalyzer.updateInterfaceStatistics()
      NetworkStatisticsBuilder.updateExistingInterfaceStatistics()``