Skip to content

feat(sql): allow list of timestamps of higher precision than required in alter table attach/detach/drop partition #3002

Merged
bluestreak01 merged 16 commits intomasterfrom
ma/drop-partition-of-higher-resolution
Feb 27, 2023
Merged

feat(sql): allow list of timestamps of higher precision than required in alter table attach/detach/drop partition #3002
bluestreak01 merged 16 commits intomasterfrom
ma/drop-partition-of-higher-resolution

Conversation

@marregui
Copy link
Contributor

@marregui marregui commented Feb 18, 2023

Closes #2879

The approach is to let the compiler only process the chars required to understand a partition dir name, while ignoring the rest, for each of the items in the comma separated list following the LIST token.

Example, for a table partitioned by month:

alter table x drop partition list '2023-03-29T23', ... ;

2023-03-29T23 is a valid partition name now for month granularity because the compiler knows that the patter for partition by month is yyyy-MM, which means it can parse a date up to month, thus -29T23 is ignored. Incidentally, this would work too alter table x drop partition list '2023-03-a-banana' ;. We are trying to make life easier for users who might use a full timestamp (select max(ts) from table) to drop a partition (the last one in this example).

WHERE is not supported, no value, and harder a problem to solve.

The case of partition by week is complicated. We first try to parse the partition name candidate with format YYYY-Www, and if it fails we assume the user sent a string that looks more like a date, or a timestamp, and we parse it with the day pattern yyyy-MM-dd, then we translate it to the timestamp of the day the week starts at.

@marregui marregui added New feature Feature requests SQL Issues or changes relating to SQL execution labels Feb 18, 2023
@marregui marregui self-assigned this Feb 18, 2023
@marregui marregui changed the title feat(sql): allow list of timestamps of higher precision than required int alter table attach/detach/drop partition feat(sql): allow list of timestamps of higher precision than required in alter table attach/detach/drop partition Feb 21, 2023
@marregui marregui requested a review from puzpuzpuz February 24, 2023 23:10
Copy link
Contributor

@puzpuzpuz puzpuzpuz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing this feature!

@ideoma
Copy link
Collaborator

ideoma commented Feb 27, 2023

[PR Coverage check]

😍 pass : 78 / 80 (97.50%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/PartitionBy.java 49 51 96.08%
🔵 io/questdb/griffin/engine/functions/date/WeekOfYearFunctionFactory.java 12 12 100.00%
🔵 io/questdb/std/datetime/microtime/TimestampFormatUtils.java 10 10 100.00%
🔵 io/questdb/griffin/SqlCompiler.java 7 7 100.00%

@bluestreak01 bluestreak01 merged commit 5148cd6 into master Feb 27, 2023
@bluestreak01 bluestreak01 deleted the ma/drop-partition-of-higher-resolution branch February 27, 2023 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New feature Feature requests SQL Issues or changes relating to SQL execution

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

ALTER TABLE DROP PARTITION should automatically truncate the given timestamp to the partition timestamp

4 participants