I read that as unambigously saying that initialised-or-not is a boolean value. If a counter were involved, the documentation would include a requirement for calls to be balanced.

The wording is trying to do some sort of balancing between initialize and shutdown which is confusing. I think the correct wording is:

shutdown does nothing unless initialize has been called explicitly or implicitly (auto init).

But even that is confusing. Perhaps no references to initialize are needed.

shutdown frees all memory and data structures that have been allocated by SQLite.

Any of these interpretations mean that CPython code is very wrong.

Any of these interpretations mean that CPython code is very wrong.

Unless the module includes its own statically linked copy of SQLite, which (checking...) it seems to do, at least on macOS.

That won't help. You can load the module multiple times in the same process. Specifically Python allows sub-interpreters where each sub-interpreter is independent of the others.¹ The sqlite3 module is marked as safe for use in sub-interpreters which means the module initialization function I showed gets run in each sub-interpreter where it is imported.

That means this flow can happen in the same process:

• Module load in interpreter 1
  • Calls initialize
  • Opens connections, does work, all is good
• Module load in interpreter 2
  • Calls initialize
  • Python things fail
  • Error handling calls shutdown

The issue is whether that shutdown will clobber the global SQLite allocations and state - ie will interpreter 1 now have dangling pointers to freed memory?

The code has been written as if providing the explicit initialize and shutdown calls are balanced, the above will be ok. The documentation sort of implies it hence this thread, but the code definitely does not to my eyes. I think the above scenario will be catastrophic to interpreter 1, but would like confirmation before filing the bug, and ideally the documentation being clarified.

1. ^{^} There is some back door very limited data passing available.

In that case, the module code must do its own counting and only call sqlite3_shutdown if the number of active module instances is zero.

As a followup the sqlite3_shutdown call in CPython has been removed.

The documentation for sqlite3_shutdown should include a copy of the comment that is above the function in the source code, which would clarify its usage.

** Must not be called while
** there are outstanding database connections or memory allocations or
** while any part of SQLite is otherwise in use in any thread.  

Bump on this documentation issue. I believe the comment from the source should start the sqlite3_shutdown doc:

Must not be called while there are outstanding database connections or memory allocations or while any part of SQLite is otherwise in use in any thread.

Do I understand correctly that there is insufficient emphasis on this point? Or is there actually an error in the documentation?

Insufficient emphasis. The emphasis is in the comments in the source but not in the doc. It should be very clear how harmful shutdown could be.

What happened is that developers wrote code like the following because they were using SQLITE_OMIT_AUTOINIT as recommended.

sqlite3_initialise();

if (do_some_initilisation() != ERROR)
    return;

sqlite3_shutdown();

The doc does go on about effective calls and balancing of initialise and shutdown, which is why the pattern above seemed sensible to them. What they hadn't realised is that other threads and multiple loading of their module meant shutdown could have catastrophic consequences, and it was hard for me to point out why.

Some time ago I used to recommend the use of shutdown, and the dev team got back to me.

SQLite has been re-written since the API was originally created. It is no longer required or recommended to use any API call to shutdown SQLite. The programmer should close all SQLite connections using sqlite3_close_v2() or one of its variations. Once this is completed, SQLite will have released all resources.

Note that various things can make sqlite3_close_v2() not complete, including, but not limited to, unfinalized statements, files being moved while SQLite has them open. You should be checking the result returned by your calls to close connections.

It is no longer required or recommended to use any API call to shutdown SQLite.

There are cases where it's necessary, though:

For most configuration options, the sqlite3_config() interface may only be invoked prior to library initialization using sqlite3_initialize() or after shutdown by sqlite3_shutdown().

The shell, for example, has to call sqlite3_shutdown() as part of its startup process on some Windows builds (search shell.c for "_shutdown").

High-level wrappers frequently need to do things which make subsequent calls to sqlite3_config() illegal, and thereby also need (if they expose sqlite3_config()) to either shut down the library at the end of their own initialization (which may not be feasible, as it may undo things like unregister VFSes installed by the wrapper), or expose sqlite3_shutdown() for clients to call (with all of its caveats).

It was correct and well intentioned when the code was originally written.

The recommended compile time options point 12 makes a good case for SQLITE_OMIT_AUTOINIT and a single manual call to sqlite3_initialize. The vast majority of sqlite3_config options can't be changed once sqlite3_initialize has been called. Python builds its own SQLite library on Windows and MacOS with SQLITE_OMIT_AUTOINIT - the system library is used on other platforms. This was based on the recommendations and allowed sqlite3_config to be called.

The loading logic for the Python sqlite3 module is:

• Create module object
• Call sqlite3_initialize and check for success
• Attach various constants, exception classes, and other paraphernalia to the module (these all require memory allocations)
• On success, return the module
• On failure free the incomplete module, and call sqlite3_shutdown

You can see how initialize and shutdown being on the same doc page, and all the talk of being "effective" (still no idea what that means), together with a desire for module loading to leave things as they found them on error lead to this code. In reality the memory allocations are a few kilobytes at most so loading never failed. Also the was no concern over any non-Python code in the same process using SQLite as the doc page doesn't warn about shutdown like its comments do.

Later on support for multiple interpreters in the same process was added, which means failure to initialize in the second would destroy what already happened in the first, although again that "effective" talk on the doc page is confusing. Also sqlite3_shutdown is not called when the module is unloaded, although it is very rare for code to do that.