Skip to content

Set SQLITE_LIMIT_FUNCTION_ARG to SQLite's normal default.#2493

Merged
kentonv merged 1 commit intomainfrom
kenton/sqlite-arg-limit
Aug 9, 2024
Merged

Set SQLITE_LIMIT_FUNCTION_ARG to SQLite's normal default.#2493
kentonv merged 1 commit intomainfrom
kenton/sqlite-arg-limit

Conversation

@kentonv
Copy link
Copy Markdown
Member

@kentonv kentonv commented Aug 7, 2024

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412
@kentonv kentonv requested review from elithrar and geelen August 7, 2024 18:23
@kentonv kentonv requested review from a team as code owners August 7, 2024 18:23
@justin-mp
Copy link
Copy Markdown
Contributor

Just to add color here, according to https://www.sqlite.org/limits.html (search for "Maximum Number Of Arguments On A Function"), the default is 100 and absolute maximum is 127 because this value is sometimes stored in an unsigned char.

Sure enough, sqliteLimit.h has:

#ifndef SQLITE_MAX_FUNCTION_ARG
# define SQLITE_MAX_FUNCTION_ARG 127
#endif

I can't seem to figure out where the default of 100 is set.

sqlite3_limit(db, SQLITE_LIMIT_COMPOUND_SELECT, 5);
sqlite3_limit(db, SQLITE_LIMIT_VDBE_OP, 25000);
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/default/maximum/ ?

@kentonv
Copy link
Copy Markdown
Member Author

kentonv commented Aug 7, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

My intent was to use the default.

I guess I can try to find out what sqlite actually reports its own default as being...

@justin-mp
Copy link
Copy Markdown
Contributor

justin-mp commented Aug 7, 2024

Digging more: the default is 127 and https://www.sqlite.org/limits.html is buggy. Here's what my sqlite3 reports:

sqlite> .limit
              length 1000000000
          sql_length 1000000000
              column 2000
          expr_depth 1000
     compound_select 500
             vdbe_op 250000000
        function_arg 127
            attached 10
 like_pattern_length 50000
     variable_number 250000
       trigger_depth 1000
      worker_threads 0

The limits from the compile-time constants are initialized into a static const int aHardLimit[] and then that array is copied into the per-database limit in openDatabase(...):

  assert( sizeof(db->aLimit)==sizeof(aHardLimit) );
  memcpy(db->aLimit, aHardLimit, sizeof(db->aLimit));

So it's the default and maximum.

sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
// because there are too many valid use cases for large argument lists, especially json_object.
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 127);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this might change in the future, maybe we should add a link to the existing documentation?

@anonrig
Copy link
Copy Markdown
Contributor

anonrig commented Aug 8, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

PS: https://www.sqlite.org/c3ref/create_function.html Also mentions max 127 value.

@kentonv
Copy link
Copy Markdown
Member Author

kentonv commented Aug 9, 2024

OK seems like this is correct as-is, just gonna merge.

@kentonv kentonv merged commit e0c1e78 into main Aug 9, 2024
@kentonv kentonv deleted the kenton/sqlite-arg-limit branch August 9, 2024 21:27
@HendrixString
Copy link
Copy Markdown

@kentonv Hi,

when is this gonna be released up stream into production ?

Unfortunately, i am hitting the upper limit of 32 (I have 36 params in json_object)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error: too many arguments on function JSON_OBJECT - 🐛 Bug Report — Runtime APIs

4 participants