Skip to content

Aurora DSQL support#903

Merged
gfx merged 2 commits intosqldef:masterfrom
sorah:dsql
Oct 26, 2025
Merged

Aurora DSQL support#903
gfx merged 2 commits intosqldef:masterfrom
sorah:dsql

Conversation

@sorah
Copy link
Contributor

@sorah sorah commented Oct 23, 2025

Completes #716

This patch adds 2 dialect support for Amazon Aurora DSQL.

  • postgres: add CREATE INDEX ASYNC syntax
    • fallback to generic parser when go_pgquery fails to parse (this dialect fails to parse) during export
  • postgres: new configuration disable_ddl_transaction flag to exclude all DDLs from transaction.

@sorah sorah force-pushed the dsql branch 2 times, most recently from 7586307 to 7268b4d Compare October 24, 2025 08:19
@sorah sorah marked this pull request as ready for review October 24, 2025 08:19
@gfx
Copy link
Contributor

gfx commented Oct 25, 2025

Awesome! Thank you for you contribution; I am not the owner of the project, but let me share my thoughts for Aurora DB supports:

Looks like it's a variant of PostgreSQL's CREATE INDEX CONCURRENTLY, and there's aready an configuration nob to add CONCURRENTLY to create indexes: create_index_concurrently: true (cf. https://github.com/sqldef/sqldef/blob/master/cmd-psqldef.md#create-index-concurrently).

So how about to add flavor: aurora to tweak generated DDLs to match the Aurora dialect? That is, when create_index_concurrently: true with flavor: aurora, psqldef generates CREATE INDEX ASYNC. Also would be nice add create_index_async: true as just an alias to create_index_concurrently.

Supporting ASYNC keywords in DDL souds great as well, as the users of Aurora would want to write it in desired DDLs.

Regadting the transaction separations, CREATE INDEX CONCURRENTLYs are already executed outside of transactions. However, transaction separations, or no transaction mode, is requrested in #656 as well, so I'll vote up for the feature.

@k0kubun what do you think?

@gfx gfx added psqldef Bugs or feature requests related to PostgreSQL minor When lableled to a pull-request, `tagpr` bumps the minor version labels Oct 25, 2025
@k0kubun
Copy link
Collaborator

k0kubun commented Oct 25, 2025

So how about to add flavor: aurora to tweak generated DDLs to match the Aurora dialect? That is, when create_index_concurrently: true with flavor: aurora, psqldef generates CREATE INDEX ASYNC. Also would be nice add create_index_async: true as just an alias to create_index_concurrently.

The design would prevent Aurora users from using CREATE INDEX CONCURRENTLY if flavor: aurora gets used for other purposes and they rely on it. How about just adding a separate flag create_index_async to add ASYNC, not as an alias to create_index_concurrently?

Regadting the transaction separations, CREATE INDEX CONCURRENTLYs are already executed outside of transactions. However, transaction separations, or no transaction mode, is requrested in #656 as well, so I'll vote up for the feature.

Sounds good.


P.S.

I am not the owner of the project

You're literally an owner of this organization. Given the amount of contributions you've made to sqldef, I have no problem with you acting as an owner of this project.

@sorah
Copy link
Contributor Author

sorah commented Oct 25, 2025

Note that DSQL is a different product and has unique dialect and characteristics, unlike traditional Aurora instances which is based on MySQL and PostgreSQL. Therefore if we would add flavor option, that must be named ‘dsql’.

Generator option to add ‘ASYNC’ flag should work, but IIRC ASYNC flag is retained on exports - I need to double check later

@gfx
Copy link
Contributor

gfx commented Oct 26, 2025

@sorah

Ah, I misunderstood ... I see, Aurora DSQL is different from Aurora Pg nor Aurora MySQL, and only Aurora DSQL supports CREATE INDEX ASYNC. So even though sqldef may introduce flavor: aurora-* in the future, it's not now, right?

Okay, create_index_async (not the alias to create_index_cuncurrently) may be useful because of my expecience that CREATE INDEXes without CONCURRENTLY resulted in serious performance degradations, but it's a separate concern.

LGTM.

@k0kubun

You're literally an owner of this organization. Given the amount of contributions you've made to sqldef, I have no problem with you acting as an owner of this project.

🙆‍♀️ 🌟

@gfx gfx merged commit 7270159 into sqldef:master Oct 26, 2025
24 checks passed
@sqldef-bot sqldef-bot bot mentioned this pull request Oct 23, 2025
@gfx
Copy link
Contributor

gfx commented Oct 26, 2025

@sorah

I'm thinking about CI -- Now that a slightly different dialect of PostgreSQL has been introduced without tests, we'd better setting up CI for Aurora DSQL. Are you interested in it?

It's really important because I am planning to migrate psqldef's parser from pgquery to the generic parser (e.g. preliminary survey: #868).

@sorah
Copy link
Contributor Author

sorah commented Oct 28, 2025

Okay, create_index_async (not the alias to create_index_cuncurrently) may be useful because of my expecience that CREATE INDEXes without CONCURRENTLY resulted in serious performance degradations, but it's a separate concern.

After I told you all here need to double check later earlier, this option was determined possible. so I was preparing the another version of this patch with this direction… but now the initial patch has been merged (thanks!) I am happy with either way. Maybe having generator option is much simpler in term of parser.


I'm thinking about CI -- Now that a slightly different dialect of PostgreSQL has been introduced without tests, we'd better setting up CI for Aurora DSQL. Are you interested in it?

It'd be helpful, but how do we? Wondering about who owes cost, and how to establish AWS access in pull requests.

@gfx
Copy link
Contributor

gfx commented Oct 28, 2025

It'd be helpful, but how do we? Wondering about who owes cost, and how to establish AWS access in pull requests.

I think there are two ways (in any ways, you must have an admin permisson of this repository):

  1. requesting AWS to get credits for maintain this project
  2. using only free tier -- IIUC Aurora DSQL has free tier

The latter may be good enough, but I can't imagine how much trouble it will be (setting up a dedicated AWS account, OIDC, Aurora DSQL, and so on ...).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minor When lableled to a pull-request, `tagpr` bumps the minor version psqldef Bugs or feature requests related to PostgreSQL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants