# Incremental Backup and Restore in C

> Incremental online backup can be performed while normal database activity proceeds uninterrupted. While the library level backup is the recommended method of implementation, the eXtremeDB runtime exports low level APIs to allow applications to provide a custom backup implementation.

> ### For alternative data export options see page Data Export and Import in C

> ## Library Level Backup

> The library level is a convenient way to integrate a database backup process into C applications. This approach assumes that the backup is done to a file system. The backup and restore functions are provided in the `mcobackup` library. Please see the Incremental Backup Implementation page for a detailed description of how the backup process is performed. Also, please see the SDK sample 18-backup_online for implementation details.

> In order to implement a full or incremental backup, the application must first define filenames for the backup files and labels for backup "tags" then set the appropriate parameters in the `mco_db_params_t` structure passed to `mco_db_open_dev()`. For example:

>         char * filename = "MyDb.bak";
>         char * label_full = "Full";
>         char * label_inc1 = "Inc1";
>         char * label_inc2 = "Inc2";

>         db_params.backup_map_size = 4 * DATABASE_SEGMENT_SIZE / DISK_PAGE_SIZE / 16;
>         db_params.backup_max_passes = 10;
>         db_params.backup_min_pages = 10;
>         sprintf(db_params.backup_map_filename, "%s","Backup_map");
>         db_params.mode_mask = MCO_DB_INCREMENTAL_BACKUP;

>         rc = mco_db_open_dev(db_name, MyDb_get_dictionary(), &dev, 1, &db_params );

> The backup map is used to keep track of database pages processed by the backup algorithm. For persistent databases the backup map is written to a file when the `mco_db_close()` API is called. If no `backup_map_filename` is specified, the default name of `<database_name>.bm` is used.

> The backup is an iterative process controlled by the two conditions `backup_max_passes` and `backup_min_pages.`

> backup_map_size | The size of the backup counters array in bytes. It must be power of two. This parameter is ignored and calculated automatically if the parameter `disk_max_database_size` is defined (non-zero)
> backup_min_pages | The minimum number of modified pages required to perform an incremental backup. When the number of remaining pages is less than `backup_min_pages` the backup process locks the database exclusively and finalizes the backup. If the number of modified pages is small, it takes a very short time to write them out and complete the backup. (Note that a value of zero disables the backup.)
> backup_max_passes | Irrespective of `backup_min_pages`, this parameter represents the maximum number of loops until an incremental backup is performed.

> A single backup file can contain several independent backups, possibly of different types (full or incremental) or with different tags or timestamps. Each backup is "framed" within a header and a footer (this frame is called a backup record).

> To create a backup file, applications call `mco_backup_create()` specifying a `file_name`, a `label` and the backup `type` `MCO_BACKUP_TYPE_SNAPSHOT`. This creates the initial backup record. Then successive incremental backup records can be added by calling `mco_backup_create()`with backup type `MCO_BACKUP_TYPE_INCREMENTAL`. Or the backup type `MCO_BACKUP_TYPE_AUTO` can be specified to cause automatic selection of a snapshot or incremental record depending on the content of the backup file; if there is no snapshot in the file yet, the backup process will create one; otherwise a partial (incremental) backup record is created.

> ## Restoring or Verifying a Backup

> Once created, the backup file can be used to restore the database from a specific backup record by calling the `mco_backup_restore()` API with the appropriate label. To view the records in a backup file call the `mco_backup_list()` API which displays the `name`, `type`, `label` and other properties of the backup records. It may sometimes be useful to check backup files for consistency without performing the restore. The `mco_backup_verify()` function is provided for this purpose.

> The following C application code snippets demonstrate how these APIs might be used. First, to backup a database:

>         {
>             mco_db_h con;
>             MCO_RET rc;
>             char * filename = “backup.bak”;
>             char * label = “a label”;

>             /* connect to a database */
>             rc = mco_db_connect( db_name, &con );

>             /* backup the database */
>             rc = mco_backup_create( con, filename, label,
>                             MCO_BACKUP_TYPE_AUTO, 1, 0, 0);
>         }

> Then to restore a database from the backup file:

>         {
>             mco_db_h con;
>             MCO_RET rc;
>             char * filename = “backup.bak”;
>             char * label = “a label”;

>             /* connect to a database */
>             rc = mco_db_connect( db_name, &con );

>             /* restore the database */
>             rc = mco_backup_restore( con, filename, label, 0, 0);
>         }

> ## Library Level API Reference:

> See page Incremental Backup C API for detailed descriptions and examples of the library level APIs.

> ## Low level C API backup functions

> The eXtremeDB backup services can be overridden via custom callback function implementations. Also, services can be turned on/off through the internal runtime configuration file `target/mcolib/mcocfg.h` define `MCO_CFG_BACKUP`.

> By default, the backup services are enabled and the following backup-related APIs are exported:

>         typedef MCO_RET (*mco_backup_info_proc_t)( uint4 phase,
>             const mco_backup_info_t * info, void * param );

>         typedef MCO_RET (*mco_backup_chunk_proc_t)( uint4 chunk_id,
>             const char * mem, uint4 mem_sz, void * param );

>         typedef MCO_RET (*mco_restore_chunk_proc_t)( uint4* chunk_id,
>             char ** mem, uint4 * mem_sz, void * param );

> The `mco_backup_info_proc_t()` and `mco_backup_chunk_proc_t()` callbacks are called by internal eXtremeDB runtime function `mco_backup()`. The first is intended to store informational headers that frame a backup record. The second stores a chunk of data. The `mco_restore_chunk_proc_t()` callback is called by internal eXtremeDB runtime function `mco_restore()` and is intended to restore a chunk of data from the backup storage. Applications cannot call these callback APIs directly.

> This core-level incremental backup/restore API does not limit the application to use a file system or any other specific type of storage. It is up to the application to decide how to store its backups. Internally the runtime functions `mco_backup()` and `mco_restore()`assure the integrity of the data chunks. Optionally the application may encrypt or compress the data in the process.

> The restore procedure requires a database with the same database properties (such as size, memory page size etc.) and overwrites its content with the data pages read from the backup record. As with the library-level backup, it is necessary to have at least one snapshot (full backup record), and any possible number of incremental records, to restore the content of a database.

> The following code snippet demonstrates one possible simple implementation of these callback APIs:

>         /* backup callbacks */
>         MCO_RET mco_backup_info(uint4 phase, const mco_backup_info_t* info, void* param)
>         {
>             mco_backup_ctx_t *ctx = (mco_backup_ctx_t *)param;
>             mco_backup_info_t hdr = *info;

>             switch (phase)
>             {
>                 case MCO_BACKUP_PHASE_1:
>                     …
>                     /* write backup header */
>                     if (fwrite(&hdr, sizeof(hdr), 1, ctx->f) != 1)
>                     {
>                         return MCO_E_DISK_WRITE;
>                     }
>                     ...
>                     break;

>                 case MCO_BACKUP_PHASE_2:
>                     ...
>                     /* write backup footer */
>                     if (fwrite(&hdr, sizeof(hdr), 1, ctx->f) != 1)
>                     {
>                         return MCO_E_DISK_WRITE;
>                     }
>                     ...
>                     break;

>                 default:
>                     return MCO_E_ILLEGAL_PARAM;
>             }

>             return MCO_S_OK;
>         }

>         MCO_RET mco_backup_chunk(uint4 chunk_id, const char* chunk, uint4 chunk_size,
>                         void* param )
>         {
>             mco_backup_ctx_t *ctx = (mco_backup_ctx_t *)param;
>             ...

>             /* write the chunk */
>             if (fwrite( chunk, chunk_size, 1, ctx->f) != 1)
>             {
>                 return MCO_E_DISK_WRITE;
>             }
>             ...
>             return MCO_S_OK;
>         }

>         MCO_RET mco_backup_create(mco_db_h con, char const* file_name, char const* label,
>                         mco_backup_type type, char* err_buf, unsigned int * err_buf_sz)
>         {
>             MCO_RET rc;
>             mco_backup_ctx_t ctx;
>             ...

>             /* open a backup file */
>             ctx.f = fopen(file_name, "rb+”);
>             ...

>             /* run backup procedure passing-in two callbacks */
>             rc = mco_backup(con, label, type, mco_backup_info, mco_backup_chunk, &ctx);
>             ...

>             /* close backup file */
>             fclose(ctx.f);
>             ...

>             return rc;
>         }

>         MCO_RET mco_restore_chunk(uint4* chunk_id, char** pchunk, uint4* pchunk_size, void * param)
>         {
>             mco_backup_ctx_t *ctx = (mco_backup_ctx_t *) param;
>             ...

>             /* read chunk into a buffer and return read size and the content */
>             ...

>             if ( ((*pchunk_size)=fread( ((*pchunk)=buf), size, 1, ctx->f)) != 1)
>             {
>                 return MCO_E_DISK_READ;
>             }

>             ...
>             return MCO_S_OK;
>         }

>         MCO_RET mco_backup_restore(mco_db_h con, char const* file_name, char const* label,
>                         char const* cipher, char* err_buf, unsigned int * err_buf_sz)
>         {
>             MCO_RET rc;
>             mco_backup_ctx_t ctx;
>             mco_backup_info_t info;
>             ...

>             /* open a backup file */
>             ctx.f = fopen(file_name, "rb+”);
>             ...

>             /* read backup header and footer*/
>             fread( &info, sizeof(info), ctx.f );
>             ...

>             /* apply the backup from the file passing-in chunk read callback and the
>                 backup info (header & footer) */
>             rc = mco_restore(con, &info, mco_restore_chunk, &ctx);
>             ...

>             /* close backup file */
>             fclose(ctx.f);
>             ...

>             return rc;
>         }