聊聊chronos的BackupDB

序

本文主要研究一下chronos的BackupDB

BackupDB

DDMQ/carrera-chronos/src/main/java/com/xiaojukeji/chronos/db/BackupDB.java

public class BackupDB {
    private static final Logger LOGGER = LogUtils.BACKUP_RESTORE_LOGGER;

    public static final String DB_PATH_BACKUP = ConfigManager.getConfig().getDbConfig().getDbPathBackup();
    public static final String DB_PATH_RESTORE = ConfigManager.getConfig().getDbConfig().getDbPathRestore();
    private static volatile boolean backuping = false;
    private static volatile boolean restoring = false;

    public static BackupState backup() {
        if (backuping) {
            LOGGER.info("is backuping, return");
            return BackupState.BEING_BACKUP;
        }

        LOGGER.info("start backup");
        backuping = true;
        try (final BackupableDBOptions bopt = new BackupableDBOptions(DB_PATH_BACKUP);
             final BackupEngine be = BackupEngine.open(Env.getDefault(), bopt)) {

            /**
             * Captures the state of the database in the latest backup
             *
             * @param db The database to backup
             * @param flushBeforeBackup When true, the Backup Engine will first issue a
             *                          memtable flush and only then copy the DB files to
             *                          the backup directory. Doing so will prevent log
             *                          files from being copied to the backup directory
             *                          (since flush will delete them).
             *                          When false, the Backup Engine will not issue a
             *                          flush before starting the backup. In that case,
             *                          the backup will also include log files
             *                          corresponding to live memtables. The backup will
             *                          always be consistent with the current state of the
             *                          database regardless of the flushBeforeBackup
             *                          parameter.
             *
             * Note - This method is not thread safe
             *
             * @throws RocksDBException thrown if a new backup could not be created
             */
            boolean flushBeforeBackup = false;
            be.createNewBackup(RDB.DB, flushBeforeBackup);

            List<BackupInfo> backupInfos = be.getBackupInfo();
            for (int i = 0; i < backupInfos.size(); i++) {
                LOGGER.info("backupInfo[{}}, backupId:{}, timestamp:{}, size:{}, numberFiles:{}", i, backupInfos.get(i).backupId(),
                        backupInfos.get(i).timestamp(), backupInfos.get(i).size(), backupInfos.get(i).numberFiles());
            }

            return BackupState.SUCCESS;
        } catch (RocksDBException e) {
            LOGGER.error("error while backup, path:{}, err:{}", DB_PATH_BACKUP, e.getMessage(), e);
            return BackupState.FAIL;
        } finally {
            backuping = false;
            LOGGER.info("end backup");
        }
    }

    public static RestoreState restore() throws RocksDBException {
        if (restoring) {
            LOGGER.info("is restoring, return");
            return RestoreState.BEING_RESTORE;
        }

        LOGGER.info("start restore");
        restoring = true;
        RocksDB restoreDB = null;
        try (final BackupableDBOptions bopt = new BackupableDBOptions(DB_PATH_BACKUP);
             final BackupEngine be = BackupEngine.open(Env.getDefault(), bopt)) {
            // restore db from first backup

            /**
             * @param keepLogFiles If true, restore won't overwrite the existing log files
             *   in wal_dir. It will also move all log files from archive directory to
             *   wal_dir. Use this option in combination with
             *   BackupableDBOptions::backup_log_files = false for persisting in-memory
             *   databases.
             *   Default: false
             */
            boolean keepLogFiles = false;
            be.restoreDbFromLatestBackup(DB_PATH_RESTORE, DB_PATH_RESTORE, new RestoreOptions(keepLogFiles));
            // open database again.
            restoreDB = RocksDB.open(OptionsConfig.DB_OPTIONS, DB_PATH_RESTORE, CFManager.CF_DESCRIPTORS, CFManager.CF_HANDLES);

            int i = 0;
            try (RocksIterator it = restoreDB.newIterator()) {
                for (it.seekToFirst(); it.isValid(); it.next()) {
                    LOGGER.info("i:{}, key:{}, value:{}", i++, new String(it.key()), new String(it.value()));
                    if (i == 10) {
                        break;
                    }
                }
            }

            return RestoreState.SUCCESS;
        } catch (RocksDBException e) {
            LOGGER.error("error while restore, path:{}, err:{}", DB_PATH_RESTORE, e.getMessage(), e);
            return RestoreState.FAIL;
        } finally {
            if (restoreDB != null) {
                restoreDB.close();
            }

            restoring = false;
            LOGGER.info("end restore");
        }
    }
}

• BackupDB提供了两个静态方法，分别是backup及restore；backup方法通过BackupEngine的createNewBackup来进行backup；restore方法通过BackupEngine的restoreDbFromLatestBackup来进行restore

BackupEngine

rocksdbjni-5.7.2-sources.jar!/org/rocksdb/BackupEngine.java

public class BackupEngine extends RocksObject implements AutoCloseable {

  protected BackupEngine(final long nativeHandle) {
    super(nativeHandle);
  }

  public void createNewBackup(
      final RocksDB db, final boolean flushBeforeBackup)
      throws RocksDBException {
    assert (isOwningHandle());
    createNewBackup(nativeHandle_, db.nativeHandle_, flushBeforeBackup);
  }

  private native void createNewBackup(final long handle, final long dbHandle,
      final boolean flushBeforeBackup) throws RocksDBException;


  public void restoreDbFromLatestBackup(
      final String dbDir, final String walDir,
      final RestoreOptions restoreOptions) throws RocksDBException {
    assert (isOwningHandle());
    restoreDbFromLatestBackup(nativeHandle_, dbDir, walDir,
        restoreOptions.nativeHandle_);
  }

  private native void restoreDbFromLatestBackup(final long handle,
      final String dbDir, final String walDir, final long restoreOptionsHandle)
      throws RocksDBException;

  //......
}

• createNewBackup方法接收RocksDB及flushBeforeBackup参数；restoreDbFromLatestBackup接收dbDir、walDir、restoreOptions参数

小结

BackupDB提供了两个静态方法，分别是backup及restore；backup方法通过BackupEngine的createNewBackup来进行backup；restore方法通过BackupEngine的restoreDbFromLatestBackup来进行restore

doc

• carrera-chronos