write_wrapper() plugin hook for intercepting write operations

A new write_wrapper() plugin hook allows plugins to intercept and wrap database write operations. (#2636)

Plugins implement the hook as a generator-based context manager:

@hookimpl
def  write_wrapper(datasette, database, request):
    def  wrapper(conn):
        # Setup code runs before the write
        yield
        # Cleanup code runs after the write

    return wrapper

register_token_handler() plugin hook for custom API token backends

A new register_token_handler() plugin hook allows plugins to provide custom token backends for API authentication. (#2650)

This includes a backwards incompatible change: the datasette.create_token() internal method is now an async method. Consult the upgrade guide for details on how to update your code.

render_cell() now receives a pks parameter

The render_cell() plugin hook now receives a pks parameter containing the list of primary key column names for the table being rendered. This avoids plugins needing to make redundant async calls to look up primary keys. (#2641)

Other changes

• Facets defined in metadata now preserve their configured order, instead of being sorted by result count. Request-based facets added via the _facet parameter are still sorted by result count and appear after metadata-defined facets. (#2647)
• Fixed --reload incorrectly interpreting the serve command as a file argument. Thanks, Daniel Bates. (#2646)

request.form() method for POST data and file uploads

Datasette now includes a request.form() method for parsing form submissions, including handling file uploads. (#2626)

This supports both application/x-www-form-urlencoded and multipart/form-data content types, and uses a new streaming multipart parser that processes uploads without buffering entire request bodies in memory.

# Parse form fields (files are discarded by default)
form = await request.form()
username = form["username"]

# Parse form fields AND file uploads
form = await request.form(files=True)
uploaded = form["avatar"]
content = await uploaded.read()

The returned FormData object provides dictionary-style access with support for multiple values per key via form.getlist("key"). Uploaded files are represented as UploadedFile objects with filename, content_type, size properties and async read() and seek() methods.

Files smaller than 1MB are held in memory; larger files automatically spill to temporary files on disk. Configurable limits control maximum file size, request size, field counts and more.

Several internal views (permissions debug, messages debug, create token) now use request.form() instead of request.post_vars().

request.post_vars() remains available for backwards compatibility but is no longer the recommended API for handling POST data.

render_cell and foreign_key_tables extras for the JSON API

The table JSON API now supports ?_extra=render_cell, which returns the rendered HTML for each cell as produced by the render_cell plugin hook. Only columns whose rendered output differs from the default are included. (#2619)

The row JSON API also gains ?_extra=render_cell and ?_extra=foreign_key_tables extras, bringing it closer to parity with the table API.

The row JSON API now returns "ok": true in its response, for consistency with the table API.

uv run pytest with a dev= dependency group

The recommended development environment for Datasette now uses uv. You can now set up a development environment and run the test suite with just uv run pytest --- no manual virtualenv or pip install step required. (#2611)

Other changes

• Plugins that raise datasette.utils.StartupError() during startup now display a clean error message instead of a full traceback. (#2624)
• Schema refreshes are now throttled to at most once per second, providing a small performance increase. (#2629)
• Minor performance improvement to remove_infinites --- rows without infinity values now skip the list/dict reconstruction step. (#2629)
• Filter inputs and the search input no longer trigger unwanted zoom on iOS Safari. Thanks, Daniel Olasubomi Sobowale. (#2346)
• table_names() and get_all_foreign_keys() now return results in deterministic sorted order. (#2628)
• Switched linting to ruff and fixed all lint errors. (#2630)

• Fix for bug where a stale database entry in internal.db could cause a 500 error on the homepage. #2605
• Cosmetic improvement to /-/actions page. #2599

• datasette serve --default-deny option for running Datasette configured to deny all permissions by default. (#2592)
• datasette.is_client() method for detecting if code is executing inside a datasette.client request. (#2594)
• datasette.pm property can now be used to register and unregister plugins in tests. (#2595)

• Fixes an open redirect security issue: Datasette instances would redirect to example.com/foo/bar if you accessed the path //example.com/foo/bar. Thanks to James Jefferies for the fix. (#2429)
• Fixed datasette publish cloudrun to work with changes to the underlying Cloud Run architecture. (#2511)
• New datasette --get /path --headers option for inspecting the headers returned by a path. (#2578)
• New datasette.client.get(..., skip_permission_checks=True) parameter to bypass permission checks when making requests using the internal client. (#2583)

• Fixes an open redirect security issue: Datasette instances would redirect to example.com/foo/bar if you accessed the path //example.com/foo/bar. Thanks to James Jefferies for the fix. #2429
• Upgraded for compatibility with Python 3.14.
• Fixed datasette publish cloudrun to work with changes to the underlying Cloud Run architecture. #2511
• Minor upgrades to fix warnings, including pkg_resources deprecation.

This alpha introduces a major breaking change prior to the 1.0 release of Datasette concerning how Datasette's permission system works.

Permission system redesign

Previously the permission system worked using datasette.permission_allowed() checks which consulted all available plugins in turn to determine whether a given actor was allowed to perform a given action on a given resource.

This approach could become prohibitively expensive for large lists of items - for example to determine the list of tables that a user could view in a large Datasette instance each plugin implementation of that hook would be fired for every table.

The new design uses SQL queries against Datasette's internal catalog tables to derive the list of resources for which an actor has permission for a given action. This turns an N x M problem (N resources, M plugins) into a single SQL query.

Plugins can use the new permission_resources_sql(datasette, actor, action) hook to return SQL fragments which will be used as part of that query.

Plugins that use any of the following features will need to be updated to work with this and following alphas (and Datasette 1.0 stable itself):

• Checking permissions with datasette.permission_allowed() - this method has been replaced with datasette.allowed().
• Implementing the permission_allowed() plugin hook - this hook has been removed in favor of permission_resources_sql().
• Using register_permissions() to register permissions - this hook has been removed in favor of register_actions().

Consult the v1.0a20 upgrade guide for further details on how to upgrade affected plugins.

Plugins can now make use of two new internal methods to help resolve permission checks:

• datasette.allowed_resources() returns a PaginatedResources object with a .resources list of Resource instances that an actor is allowed to access for a given action (and a .next token for pagination).
• datasette.allowed_resources_sql() returns the SQL and parameters that can be executed against the internal catalog tables to determine which resources an actor is allowed to access for a given action. This can be combined with further SQL to perform advanced custom filtering.

Related changes:

• The way datasette --root works has changed. Running Datasette with this flag now causes the root actor to pass all permission checks. (#2521)
• Permission debugging improvements:
  • The /-/allowed endpoint shows resources the user is allowed to interact with for different actions.
  • /-/rules shows the raw allow/deny rules that apply to different permission checks.
  • /-/actions lists every available action.
  • /-/check can be used to try out different permission checks for the current actor.

Other changes

• The internal catalog_views table now tracks SQLite views alongside tables in the introspection database. (#2495)
• Hitting the / brings up a search interface for navigating to tables that the current user can view. A new /-/tables endpoint supports this functionality. (#2523)
• Datasette attempts to detect some configuration errors on startup.
• Datasette now supports Python 3.14 and no longer tests against Python 3.9.

• Tiny cosmetic bug fix for mobile display of table rows. #2479

• Fix for incorrect foreign key references in the internal database schema. #2466
• The prepare_connection() hook no longer runs for the internal database. #2468
• Fixed bug where link: HTTP headers used invalid syntax. #2470
• No longer tested against Python 3.8. Now tests against Python 3.13.
• FTS tables are now hidden by default if they correspond to a content table. #2477
• Fixed bug with foreign key links to rows in databases with filenames containing a special character. Thanks, Jack Stratton. #2476

• DATASETTE_SSL_KEYFILE and DATASETTE_SSL_CERTFILE environment variables as alternatives to --ssl-keyfile and --ssl-certfile. Thanks, Alex Garcia. (#2422)
• SQLITE_EXTENSIONS environment variable has been renamed to DATASETTE_LOAD_EXTENSION. (#2424)
• datasette serve environment variables are now documented here.
• The register_magic_parameters(datasette) plugin hook can now register async functions. (#2441)
• Datasette is now tested against Python 3.13.
• Breadcrumbs on database and table pages now include a consistent self-link for resetting query string parameters. (#2454)
• Fixed issue where Datasette could crash on metadata.json with nested values. (#2455)
• New internal methods datasette.set_actor_cookie() and datasette.delete_actor_cookie(), described here. (#1690)
• /-/permissions page now shows a list of all permissions registered by plugins. (#1943)
• If a table has a single unique text column Datasette now detects that as the foreign key label for that table. (#2458)
• The /-/permissions page now includes options for filtering or exclude permission checks recorded against the current user. (#2460)
• Fixed a bug where replacing a database with a new one with the same name did not pick up the new database correctly. (#2465)