Skip to content

feat(bigtable): Hot backups#11215

Merged
bhshkh merged 9 commits into
googleapis:mainfrom
bhshkh:feature/cbt-hot-backup
Jan 8, 2025
Merged

feat(bigtable): Hot backups#11215
bhshkh merged 9 commits into
googleapis:mainfrom
bhshkh:feature/cbt-hot-backup

Conversation

@bhshkh

@bhshkh bhshkh commented Dec 3, 2024

Copy link
Copy Markdown
Contributor

Fixes: b/365774730

More about hot backups: https://cloud.google.com/bigtable/docs/backups#hot-backups

Generated protos have 2 new fields in the Backup struct. These fields are being added to various methods in this PR:

// Indicates the backup type of the backup.
BackupType Backup_BackupType `protobuf:"varint,11,opt,name=backup_type,json=backupType,proto3,enum=google.bigtable.admin.v2.Backup_BackupType" json:"backup_type,omitempty"`
// The time at which the hot backup will be converted to a standard backup.
// Once the `hot_to_standard_time` has passed, Cloud Bigtable will convert the
// hot backup to a standard backup. This value must be greater than the backup
// creation time by:
// - At least 24 hours
//
// This field only applies for hot backups. When creating or updating a
// standard backup, attempting to set this field will fail the request.
HotToStandardTime *timestamppb.Timestamp `protobuf:"bytes,12,opt,name=hot_to_standard_time,json=hotToStandardTime,proto3" json:"hot_to_standard_time,omitempty"`


To create a hot backup that expires after 4 days:

hotToStdTime := 
adminClient.CreateBackupWithOptions(ctx, "TableID", "Cluster", "BkpName",
		time.Now().Add(24 * 4 * time.Hour), HotToStandardBackup(time.Now().Add(2 * 24 * time.Hour)))

or without specifying HotToStandardTime i.e. hot backup does not get converted to standard backup

hotToStdTime := time.Now()
adminClient.CreateBackupWithOptions(ctx, "TableID", "Cluster", "BkpName",
		time.Now().Add(24 * 4 * time.Hour), HotBackup())

To update hot_to_standard_time of hot backup:

adminClient.UpdateBackupHotToStandardTime(ctx, "Cluster", "BkpName", &hotToStdTime)

To remove hot_to_standard_time of hot backup:

adminClient.UpdateBackupHotToStandardTime(ctx, "Cluster", "BkpName", nil)

@bhshkh bhshkh requested review from a team December 3, 2024 20:22
@bhshkh bhshkh changed the title feat(bigtable): hot backups feat(bigtable): Hot backups Dec 3, 2024
@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the Bigtable API. label Dec 4, 2024
Comment thread bigtable/admin.go Outdated
Comment thread bigtable/admin.go Outdated

@DerekLeeCS DerekLeeCS left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall logic looks good, just have a few suggestions for the implementation

Comment thread bigtable/admin.go
Comment thread bigtable/admin.go Outdated
Comment thread bigtable/admin.go Outdated
@bhshkh bhshkh enabled auto-merge (squash) December 10, 2024 20:52
Comment thread bigtable/admin.go Outdated
}

// BackupOption can be used to specify parameters for backup operations.
type BackupOption interface {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my main question here for the option style here is how complex you expect the options to become. I see only a single option with two behavior fields, so struct-based options that satisfy the apply() contract may be more than you need.

By way of comparison, if the contract is simplified to the core backupOptions and some funcs that can modify it:

type BackupOption func(*backupOptions)

func WithHotBackupOption(t time.Time) BackupOption {
  return func(bo *backupOptions) {
     bo.backupType = blah
     bo.hotToStandardTime = blah
  }
}

func WithExpiration(t time.Time) BackupOption {
  return func(bo *backupOptions) {
     bo.expireTime = blah
  }
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. I have updated the code.

Comment thread bigtable/admin.go
@bhshkh bhshkh requested a review from shollyman December 25, 2024 23:08
@bhshkh

bhshkh commented Jan 5, 2025

Copy link
Copy Markdown
Contributor Author

@DerekLeeCS , please review. I have updated the code as per review comments

@bhshkh

bhshkh commented Jan 6, 2025

Copy link
Copy Markdown
Contributor Author

@shollyman @gkevinzheng, please review. I have updated the code as per review comments

@DerekLeeCS DerekLeeCS left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the changes!

Comment thread bigtable/admin.go Outdated
Comment thread bigtable/integration_test.go Outdated
@bhshkh bhshkh merged commit 238ac1c into googleapis:main Jan 8, 2025
@bhshkh bhshkh deleted the feature/cbt-hot-backup branch January 8, 2025 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the Bigtable API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants