Skip to content

feat: Add retention period for DB to cli commands#26520

Merged
mgattozzi merged 1 commit intomainfrom
mgattozzi/core/retention-period
Jun 16, 2025
Merged

feat: Add retention period for DB to cli commands#26520
mgattozzi merged 1 commit intomainfrom
mgattozzi/core/retention-period

Conversation

@mgattozzi
Copy link
Copy Markdown
Contributor

@mgattozzi mgattozzi commented Jun 13, 2025

This commit touches quite a few things, but the main changes that need to be taken into account are:

  • An update command has been added to the CLI. This could be further extended in the future to update more than just Database retention periods. The API call for that has been written in such a way as to allow other qualities of the database to be updated at runtime from one API call. For now it only allows the retention period to be updated, but it could in theory allow us to rename a database without needing to wipe things, especially with a stable ID underlying everything.
  • The create database command has been extended to allow its creation with a retention period. In tandem with the update command users can now assign or delete retention periods at will
  • The ability to query catalog data about both databases and tables has been added as well. This has been used in tests added in this commit, but is also a fairly useful query when wanting to look at things such as the series key. This could be extended to a CLI command as well if we want to allow users to look at this data, but for now it's in the _internal table.

With these changes a nice UX has been created to allow our customers to work with retention periods.

@mgattozzi mgattozzi requested review from hiltontj and waynr June 13, 2025 20:09
Copy link
Copy Markdown
Contributor

@waynr waynr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't look as closely at this as I did at the corresponding enterprise PR, mostly just a spot check to make sure none of the enterprise-only bits made it. I say :shipit: (after the lint failure is addressed of course).

This commit touches quite a few things, but the main changes that need
to be taken into account are:

- An update command has been added to the CLI. This could be further
  extended in the future to update more than just Database retention
  periods. The API call for that has been written in such a
  way as to allow other qualities of the database to be updated
  at runtime from one API call. For now it only allows the retention
  period to be updated, but it could in theory allow us to rename a
  database without needing to wipe things, especially with a stable ID
  underlying everything.
- The create database command has been extended to allow
  its creation with a retention period. In tandem with the update
  command users can now assign or delete retention periods at will
- The ability to query catalog data about both databases and tables has
  been added as well. This has been used in tests added in this commit,
  but is also a fairly useful query when wanting to look at things such
  as the series key. This could be extended to a CLI command as well if
  we want to allow users to look at this data, but for now it's in the
  _internal table.

With these changes a nice UX has been created to allow our customers to
work with retention periods.
@mgattozzi mgattozzi force-pushed the mgattozzi/core/retention-period branch from 900aca9 to ead1fb0 Compare June 13, 2025 21:36
@mgattozzi mgattozzi merged commit d07d2f7 into main Jun 16, 2025
12 checks passed
@mgattozzi mgattozzi deleted the mgattozzi/core/retention-period branch June 16, 2025 14:22
mgattozzi added a commit that referenced this pull request Jun 30, 2025
This adds the update command to the help text output which was an
oversight from my work in #26520.
mgattozzi added a commit that referenced this pull request Jun 30, 2025
This adds the update command to the help text output which was an
oversight from my work in #26520.
praveen-influx added a commit that referenced this pull request Jul 3, 2025
* chore: version bump in Cargo.toml to 3.3.0-nightly (#26566)

* chore: version bump in Cargo.toml to 3.3.0-nightly

* chore: update install script to point to 3.2.0 version

* chore: Update to Rust 1.88 (#26567)

* chore: Update to Rust 1.88

* chore: Fixes missed by Claude

* fix: Add help text for the new update subcommand (#26569)

This adds the update command to the help text output which was an
oversight from my work in #26520.

* fix: --object-store is explicitly marked required (#26575)

Before this commit, although `--object-store` is mandatory it is not
reflected in the error messages. The examples are listed in the issue
976.

This commit makes `object_store` explicitly required which means error
messages include `--object-store` listed as mandatory

closes: influxdata/influxdb_pro#976

* chore: updates to Dockerfile (#26576)

- update rust version to 1.88
- remove defaulting to `file` for object store

helps: influxdata/influxdb_pro#976

* fix: v1 query API should default to ns for CSV output (#26577)

* fix: Existing soft-deleted schema can be hard-deleted (#26574)

* feat: Allow hard_deleted date of deleted schema to be updated

* feat: Include hard_deletion_date in `_internal` `databases` and `tables`

* feat: Unit tests for testing deleted databases

* chore: Default is now to hard-delete with default duration

* test: Update test names and assertions for new default hard deletion behavior

- Renamed delete_table_defaults_to_hard_delete_never to delete_table_defaults_to_hard_delete_default
- Renamed delete_database_defaults_to_hard_delete_never to delete_database_defaults_to_hard_delete_default
- Updated assertions to expect default deletion duration instead of None/Never
- Aligns with the change of HardDeletionTime default from Never to Default

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Remove TODO

* chore: PR feedback and other improvements

* Ensure system databases and tables schema specify a timezone for the
  `hard_deletion_time` Timestamp columns (otherwise they display without
  a timezone)
* `DELETE` using `default` delay is idempotent, so multiple requests
  will not continue to update the `hard_deletion_time`
* Improved test coverage for these behaviours

---------

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: Michael Gattozzi <mgattozzi@influxdata.com>
Co-authored-by: wayne <wayne.warren.s@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants