feat(db): add support for non-node libsql client by Adammatthiesen · Pull Request #14204 · withastro/astro

Adammatthiesen · 2025-08-08T16:33:48Z

Changes

Closes Error when building Astro with @astrojs/db and @astrojs/cloudflare in version v5.0.8 astro:content-asset-propagation] module is not defined #13152

This PR solves the outstanding issue that was being caused by the usage of the @libsql/client's default export of their node client, by introducing a new option to switch from the node client to the web client. Allowing users on platforms like Cloudflare to finally be able to use Astro DB since the Studio sunset.

Testing

There is currently no tests testing external server connections. And im not sure how possible it would be to spin up a libsql server locally in CI for tests.

Docs

CC @sarah11918 at your request i'm tagging you!

changeset-bot · 2025-08-08T16:33:52Z

🦋 Changeset detected

Latest commit: ccdb5d3

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Adammatthiesen · 2025-08-08T16:34:05Z

CC @Fryuni does this look correct?

…eter

…ntegration

… tests

Fryuni · 2025-08-08T17:51:46Z

In theory @libsql/client automatically switches to the appropriate implementation for the runtime it is running on. So if the missing module is coming from LibSQL this is a bug there.

If it is coming from Drizzle Client's module then I don't know if it was supposed to work or not. Worth bringing it up on their discord or repo.

My understanding would be that as well... but once i looked at how libsql client "decides" which one to use.... I think i found why it doesn't work... since when your deploying to CF, its still running node underlying when doing the import.

As for drizzle docs... they have this note

defaults to node import, automatically changes to web if target or platform is set for bundler, e.g. esbuild --platform=browser

Fryuni · 2025-08-08T17:54:55Z

-import type { LibSQLDatabase } from 'drizzle-orm/libsql';
-import { drizzle as drizzleLibsql } from 'drizzle-orm/libsql';
+import { createClient as createClientWeb } from '@libsql/client/web';
+import { drizzle as drizzleLibsql, type LibSQLDatabase } from 'drizzle-orm/libsql';


Won't Cloudflare complain just by importing this module? If that is the case, we'll need to do some tricks using virtual modules to only import the modules that are safe for the runtime.

Yeah i was wondering the same thing... I'll convert the db client config parts into dedicated internal virtual modules that will be enabled depending on mode instead, here after i make my coffee.

…tions

…o utils

…sistency

fix(db): ensure local client module is returned in default case

…modules

…ql-remote tests

Adammatthiesen · 2025-08-09T10:08:23Z

OKAY... @Fryuni I think its at the point that everything is working (finally fixed the tests too), and I've virtualized all the libsql/drizzle clients... that was a bit annoying to do 😅 but should be worth it! I think its ready to be moved from draft?

Fryuni

Looking really good

Fryuni · 2025-08-09T11:50:52Z

+export function parseOpts(config: Record<string, string>): Partial<LibSQLConfig> {
+	return {
+		...config,
+		...(config.syncInterval ? { syncInterval: parseInt(config.syncInterval) } : {}),
+		...('readYourWrites' in config ? { readYourWrites: config.readYourWrites !== 'false' } : {}),
+		...('offline' in config ? { offline: config.offline !== 'false' } : {}),
+		...('tls' in config ? { tls: config.tls !== 'false' } : {}),
+		...(config.concurrency ? { concurrency: parseInt(config.concurrency) } : {}),
+	};


Can't we use Zod for this? It has coercion and transformations.

........ now that you say that.... yes... yes we could... why i didnt do that in the first place? no freaking clue...

okay got the zod transformer built and working 😄

Fryuni · 2025-08-09T11:54:41Z

 	appToken: string;
 	isBuild: boolean;
 	output: AstroConfig['output'];
+	__execute?: boolean;


Suggested change

__execute?: boolean;

// Request module to be loaded immediately in process

localExecution?: boolean;

Fryuni · 2025-08-09T11:56:14Z

+		case 'web': {
+			return `export { createClient } from '${DB_CLIENTS.web}';`;
+		}


Suggested change

case 'web': {

return `export { createClient } from '${DB_CLIENTS.web}';`;

}

case 'web':

return `export { createClient } from '${DB_CLIENTS.web}';`;

sarah11918

Woo! What an accomplishment! Quick thoughts from me...

(Also noting, I don't see any new error messages, so checking that we don't need any new specific ones, nor an update to any existing because our existing ones don't know about this option)

Thank god for the save by the docs queen Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

Adammatthiesen · 2025-08-11T12:15:38Z

Woo! What an accomplishment! Quick thoughts from me...

(Also noting, I don't see any new error messages, so checking that we don't need any new specific ones, nor an update to any existing because our existing ones don't know about this option)

Only user facing change is the new options added to DB, that previously never existed (not even an object inside the integration function) everything else is the same "internals" just now with the ability to pick web or node drivers (since for some reason the auto detection by the libsql/client doesn't work... likely related to how Astro's build pipeline works under the hood)

ematipico · 2025-08-11T13:00:22Z

+export function getLocalVirtualModContents({
+	tables,
+	root,
+	localExecution = false,
+}: {


Can you add some JSDocs that explain the business logic of the function? The new localExecution adds some logic that isn't clear to me. For example, it doesn't explain why, when localExecution is true, we need the node client. It's an assumption that isn't explained.

The node client has full protocol support, file:, libsql:, http:. and https: where as the web client is missing the file: support (for obvious reasons) So for users who utilize the file: protocol (when we are running the execute command) we need to ensure that we have the node client available.

I've extracted the duplicated logic to a helper function, and added jsdocs to the new function and the two instances of localExecution

ematipico · 2025-08-11T13:04:26Z

 	isBuild: boolean;
 	output: AstroConfig['output'];
+	// Request module to be loaded immediately in process  
+    localExecution?: boolean; 


If possible, I would avoid the ? here. Why? Because it's an internal code, and by adding ? we might miss code paths that might require true instead. This is usually a pattern I follow inside code that isn't exposed to the user: the compiler fails all the instances of the code, and forces me to update them all. Would you mind doing that here and in the other function?

Updated in latest commit

ematipico · 2025-08-11T13:07:02Z

+	const clientImport = localExecution
+		? `import { createClient } from '${DB_CLIENTS.node}';`
+		: `import { createClient } from '${VIRTUAL_CLIENT_MODULE_ID}';`;


To me, using a node client for a remote feels like an error. However, I am sure you think it's not. Is it because the execution of commands can be done only locally? If so, then I have two suggestions:

commands shouldn't call getRemoteVirtualModContents

add some JsDoc to getRemoteVirtualModContents that explains the implications of localExecution for this function

see #14204 (comment) for the context, we previously only had the node client, and that is the recommended approach by drizzle and libsql teams unless you specifically need one of the other limited exports. (as a file: protocol CAN be considered "remote")

Co-authored-by: Emanuele Stoppa <my.burning@gmail.com>

ematipico

Thank you for addressing my questions!

Adammatthiesen · 2025-08-12T17:27:08Z

So @sarah11918 I'm guessing that the best place for the new config options docs wise would be replacing the note under this section?

sarah11918 · 2025-08-13T21:38:04Z

@Adammatthiesen That note can certainly be removed!

I will note that section is about connecting to remote databases, and so maybe we don't want to clutter inside one set of instructions (connecting) with an entirely new topic (a configuration option).

I don't think the db integration currently takes any config options? We should probably now have a ## Configuration section to properly document mode like an API reference. (See the Node.js adapter as an example).

Then, in the "connect to a remote database" instructions, if something is different for 'web' vs default mode, we can explain any different steps there, and link to the mode reference as needed.

Adammatthiesen · 2025-08-14T01:47:19Z

@Adammatthiesen That note can certainly be removed!

I will note that section is about connecting to remote databases, and so maybe we don't want to clutter inside one set of instructions (connecting) with an entirely new topic (a configuration option).

I don't think the db integration currently takes any config options? We should probably now have a ## Configuration section to properly document mode like an API reference. (See the Node.js adapter as an example).

Then, in the "connect to a remote database" instructions, if something is different for 'web' vs default mode, we can explain any different steps there, and link to the mode reference as needed.

The difference is the fact that the web mode, would allow db to work on environments like CF but would restrict usage of local file paths (i.e. file://path/to/db.db is only available in node mode) which is why i do think it would be relevant on the "Connecting to Remote Databases" esp if we are talking from a deployment guide point of view.

I also agree that we will need the full config reference as well, most likely just adding to the dedicated integration page? https://docs.astro.build/en/guides/integrations-guide/db/

sarah11918 · 2025-08-14T11:56:20Z

I'm confused, the first link you posted was the db integration guide? that's the only guide in question here, right?

Adammatthiesen · 2025-08-14T15:03:34Z

I'm confused, the first link you posted was the db integration guide? that's the only guide in question here, right?

there is two pages... the guide page, and the technical page.... so maybe both... thats why im asking you after all... your the docs person who knows best. 😅

sarah11918 · 2025-08-14T15:29:49Z

Haha, sorry, I thought you were referring to the same page twice. I'll figure out this docs thing eventually...

sarah11918 · 2025-08-20T11:36:00Z

OK, so catching up on this yes, we do need this config option mentioned on the integration page itself. Typically, a ## Configuration section for configuring the integration itself would go after ## Installation. So let's create that new section and document mode as now a configuration option.

Then yes, in "connecting to remote databases" replace the note with a sub-heading, something maybe like ## Non-Node.js clients (replace "clients" with whatever the appropriate word is?). Then, add whatever text you think is needed there, and when you mention that setting mode: 'web' is required, link back to the API reference from the guide page.

Does that sound about right to you?

Adammatthiesen · 2025-08-22T05:50:32Z

OK, so catching up on this yes, we do need this config option mentioned on the integration page itself. Typically, a ## Configuration section for configuring the integration itself would go after ## Installation. So let's create that new section and document mode as now a configuration option.

Then yes, in "connecting to remote databases" replace the note with a sub-heading, something maybe like ## Non-Node.js clients (replace "clients" with whatever the appropriate word is?). Then, add whatever text you think is needed there, and when you mention that setting mode: 'web' is required, link back to the API reference from the guide page.

Does that sound about right to you?

Probably use the term "driver" instead of client? but yup sounds perfect. I'll work on the docs here in the next few days!

Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

Adammatthiesen · 2025-08-25T03:01:55Z

@sarah11918 Docs PR for this is now up withastro/docs#12208

sarah11918

Approving for docs! 🎉

Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

* feat(db): add support for non-node libsql client * feat(db): update remote database info retrieval to include mode parameter * feat(db): export AstroDBConfig type alongside integration from core integration * test(db): ensure mode parameter is passed to getRemoteDatabaseInfo in tests * feat(db): refactor database client handling and remove legacy code * test(db): remove mode parameter from getRemoteDatabaseInfo assertions * test(db): remove mode parameter from getRemoteDatabaseInfo call in tests * fix(db): ensure correct handling of 'node' mode in client module functions * refactor(db): consolidate utility functions by moving hasPrimaryKey to utils * fix(db): re-export hasPrimaryKey from utils for consistency * fix(db): update import path for hasPrimaryKey to utils for consistency * fix(db): update client import to use VIRTUAL_CLIENT_MODULE_ID for consistency * fix(db): remove commented export of hasPrimaryKey for clarity * fix(db): update DB_CLIENTS paths to use '@astrojs' for consistency fix(db): ensure local client module is returned in default case * fix(db): update DB_CLIENTS paths to use PACKAGE_NAME for consistency * fix(db): simplify db-client export paths for consistency * fix(db): unify client creation function names across db-client modules * fix(db): standardize client creation function names across db-client modules * fix(db): rename remote database client import for consistency * fix(db): change const to let for parsedUrl in createClient function * fix(db): update variable naming for consistency in createClient function * fix(test): add logging for prodDbPath and ASTRO_DB_REMOTE_URL in libsql-remote tests * fix(test): add logging for absoluteFileUrl in libsql-remote tests * fix(db): correct variable reference for rawUrl in createClient function * fix(test): remove debug logging for prodDbPath in libsql-remote tests * fix(db): rename __execute to localExecution for clarity in virtual module loading * refactor(db): replace parseOpts with parseLibSQLConfig for improved configuration handling * fix(db): prevent redundant assignment for 'url' in LibSQL config parsing * refactor(db): streamline libSQL configuration handling in createClient functions * fix(test): remove redundant 'url' field from config assertions in db-client tests * refactor(db): enhance boolean parsing logic in LibSQL config handling * Apply suggestions from code review Thank god for the save by the docs queen Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Emanuele Stoppa <my.burning@gmail.com> * Refactor localExecution handling in vitePluginDb and related functions * Update .changeset/full-dingos-repeat.md Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com> * Update .changeset/full-dingos-repeat.md Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com> --------- Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com> Co-authored-by: Emanuele Stoppa <my.burning@gmail.com>

feat(db): add support for non-node libsql client

a66455f

Adammatthiesen added 3 commits August 8, 2025 09:42

feat(db): update remote database info retrieval to include mode param…

2ddea20

…eter

feat(db): export AstroDBConfig type alongside integration from core i…

4311aa6

…ntegration

test(db): ensure mode parameter is passed to getRemoteDatabaseInfo in…

6ad1ac8

… tests

Fryuni reviewed Aug 8, 2025

View reviewed changes

Adammatthiesen added 21 commits August 8, 2025 20:11

feat(db): refactor database client handling and remove legacy code

e34acea

test(db): remove mode parameter from getRemoteDatabaseInfo assertions

12a4379

test(db): remove mode parameter from getRemoteDatabaseInfo call in tests

d9bd4b1

fix(db): ensure correct handling of 'node' mode in client module func…

322b7cf

…tions

refactor(db): consolidate utility functions by moving hasPrimaryKey t…

475c624

…o utils

fix(db): re-export hasPrimaryKey from utils for consistency

604648e

fix(db): update import path for hasPrimaryKey to utils for consistency

4187ecc

fix(db): update client import to use VIRTUAL_CLIENT_MODULE_ID for con…

f3572c8

…sistency

fix(db): remove commented export of hasPrimaryKey for clarity

be17d1e

fix(db): update DB_CLIENTS paths to use '@astrojs' for consistency

2a4f737

fix(db): ensure local client module is returned in default case

fix(db): update DB_CLIENTS paths to use PACKAGE_NAME for consistency

4b661ab

fix(db): simplify db-client export paths for consistency

d00fc28

fix(db): unify client creation function names across db-client modules

5b2c056

fix(db): standardize client creation function names across db-client …

3bc6eac

…modules

fix(db): rename remote database client import for consistency

24a49b2

fix(db): change const to let for parsedUrl in createClient function

92516ee

fix(db): update variable naming for consistency in createClient function

c9e5592

fix(test): add logging for prodDbPath and ASTRO_DB_REMOTE_URL in libs…

736b2d2

…ql-remote tests

fix(test): add logging for absoluteFileUrl in libsql-remote tests

03892a8

fix(db): correct variable reference for rawUrl in createClient function

95bf697

fix(test): remove debug logging for prodDbPath in libsql-remote tests

95dcef0

Fryuni reviewed Aug 9, 2025

View reviewed changes

Merge branch 'main' into main

013d98a

Adammatthiesen marked this pull request as ready for review August 10, 2025 14:16

sarah11918 reviewed Aug 11, 2025

View reviewed changes

Comment thread .changeset/full-dingos-repeat.md Outdated

Comment thread .changeset/full-dingos-repeat.md Outdated

Comment thread .changeset/full-dingos-repeat.md Outdated

Apply suggestions from code review

f0311bc

Thank god for the save by the docs queen Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

ematipico reviewed Aug 11, 2025

View reviewed changes

Adammatthiesen and others added 2 commits August 11, 2025 06:20

Apply suggestions from code review

9fde9ec

Co-authored-by: Emanuele Stoppa <my.burning@gmail.com>

Refactor localExecution handling in vitePluginDb and related functions

5604a6f

ematipico approved these changes Aug 11, 2025

View reviewed changes

sarah11918 reviewed Aug 22, 2025

View reviewed changes

Comment thread .changeset/full-dingos-repeat.md Outdated

Adammatthiesen mentioned this pull request Aug 25, 2025

Update docs for new AstroDB update withastro/docs#12208

Merged

Update .changeset/full-dingos-repeat.md

c45022e

Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

ematipico added this to the v5.14.0 milestone Aug 25, 2025

Fryuni approved these changes Sep 5, 2025

View reviewed changes

sarah11918 reviewed Sep 5, 2025

View reviewed changes

Comment thread .changeset/full-dingos-repeat.md

sarah11918 approved these changes Sep 5, 2025

View reviewed changes

Adammatthiesen and others added 2 commits September 5, 2025 20:13

Update .changeset/full-dingos-repeat.md

781281b

Co-authored-by: Sarah Rainsberger <5098874+sarah11918@users.noreply.github.com>

Merge branch 'main' into main

ccdb5d3

Adammatthiesen merged commit d71448e into withastro:main Sep 11, 2025
18 of 19 checks passed

astrobot-houston mentioned this pull request Sep 11, 2025

[ci] release #14342

Merged

	__execute?: boolean;
	// Request module to be loaded immediately in process
	localExecution?: boolean;

Uh oh!

Conversation

Adammatthiesen commented Aug 8, 2025

Changes

Testing

Docs

Uh oh!

changeset-bot Bot commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Adammatthiesen commented Aug 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Adammatthiesen commented Aug 9, 2025

Uh oh!

Fryuni left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarah11918 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Adammatthiesen commented Aug 11, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ematipico left a comment

Choose a reason for hiding this comment

Uh oh!

Adammatthiesen commented Aug 12, 2025

Uh oh!

sarah11918 commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Adammatthiesen commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sarah11918 commented Aug 14, 2025

Uh oh!

Adammatthiesen commented Aug 14, 2025

Uh oh!

sarah11918 commented Aug 14, 2025

Uh oh!

sarah11918 commented Aug 20, 2025

Uh oh!

changeset-bot Bot commented Aug 8, 2025 •

edited

Loading

sarah11918 commented Aug 13, 2025 •

edited

Loading

Adammatthiesen commented Aug 14, 2025 •

edited

Loading