Skip to content

Add SQLite migration to fix mapzones having NULL id for all rows#1177

Merged
rtldg merged 6 commits intoshavitush:masterfrom
jedso:mapzones-rowid-fix
Oct 8, 2022
Merged

Add SQLite migration to fix mapzones having NULL id for all rows#1177
rtldg merged 6 commits intoshavitush:masterfrom
jedso:mapzones-rowid-fix

Conversation

@jedso
Copy link
Contributor

@jedso jedso commented Oct 1, 2022

SQLite uses ROWID to uniquely identify each row in a table, and a column with type INTEGER PRIMARY KEY becomes an alias for that ROWID value. The CREATE TABLE statement for mapzones since the beginning (1.1b) has been id INT AUTO_INCREMENT, but the column has to be of INTEGER type for it to act as a ROWID alias (not INT). Because id is therefore acting as a ordinary integer table column, all id rows in mapzones have been NULL in SQLite DBs since the plugin was released.

This migration fixes that and gives each map zone a unique identifier for SQLite DBs.

@jedso
Copy link
Contributor Author

jedso commented Oct 2, 2022

Just had a thought that SQLite DBs created prior to this PR with an empty mapzones table will always return 0 for SELECT EXISTS(SELECT 1 FROM %smapzones WHERE id IS NULL); regardless of whether the table is using id INT AUTO_INCREMENT, so it's probably better to use the SELECT sql FROM sqlite_master + StrContains approach from #1176.

}

char sQuery[256];
FormatEx(sQuery, sizeof(sQuery), "SELECT EXISTS(SELECT 1 FROM `%smapzones` WHERE id IS NULL);", gS_SQLPrefix);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would be fine to replace this with SELECT 1 FROM mapzones WHERE id IS NULL LIMIT 1;?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the original query, this doesn't handle the scenario I mentioned above (empty mapzones table with the old table creation statement). So ideally you'd need to directly test id's type to definitively know if the migration is required or not.

You could theoretically replace the query using a PRAGMA function, but it is listed as experimental so might break in the future:

Suggested change
FormatEx(sQuery, sizeof(sQuery), "SELECT EXISTS(SELECT 1 FROM `%smapzones` WHERE id IS NULL);", gS_SQLPrefix);
FormatEx(sQuery, sizeof(sQuery), "SELECT 1 FROM pragma_table_info('%smapzones') WHERE name = 'id' AND type = 'INT AUTO_INCREMENT' LIMIT 1;", gS_SQLPrefix);

This is the only way of accessing PRAGMA results with a SELECT statement (i.e. SELECT * FROM PRAGMA table_info(mapzones) WHERE ...; doesn't work). Then again, the docs do say that "specific pragma statements may be removed and others added in future releases of SQLite. There is no guarantee of backwards compatibility" so probably avoiding PRAGMA statements and functions is the best move long term.

Which means, as far as I can tell, the best way of testing id's type is to use SELECT sql FROM sqlite_master + StrContains as I did in #1176. Let me know if you agree with that approach and I'll commit it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if you agree with that approach and I'll commit it.

I didn't see this, sorry. Yeah that definitely seems to be a good/easy approach

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, done. Should be good now.

LogError("Timer error! SQLiteMapzonesROWID migration failed. Reason: %s", error);
return;
}
else if (!results.FetchInt(0)) // No NULL ids
Copy link
Collaborator

@rtldg rtldg Oct 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs results.FetchRow() to be called first.
In fact, since .FetchRow() returns a bool if a row was fetched, it should be fine to swap it to this:

Suggested change
else if (!results.FetchInt(0)) // No NULL ids
else if (!results.FetchRow()) // No NULL ids

Edit: Actually, this suggestion is fine to commit as it if the other thing for SELECT 1 FROM mapzones WHERE id IS NULL LIMIT 1; is done.
Otherwise, it'd need if (!results.FetchRow() || !results.FetchInt(0)) or something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah good catch. Made me wonder why I didn't get an error during testing, and it turns out that up until recently the SQLite driver did not enforce calling FetchRow() before accessing the first row. I'm still on SM 1.10.0.6545 which doesn't have the fix.

@rtldg
Copy link
Collaborator

rtldg commented Oct 3, 2022

I'm not a fan of doing the char SQLiteMapzonesQuery[1024]; global variable thing but that's probably the easiest way that won't require changing much else like wrapping things in functions or whatever.
Also not that it really matters, but I think it's funny to realize that the bool gB_MigrationsApplied[255]; uses the same amount of memory since bools are cell sized while chars are byte-sized but packed into cell-sized divisible memory.

@rtldg
Copy link
Collaborator

rtldg commented Oct 6, 2022

I resolved the conflicts on the branch since I just merged the other fk pr. So note to self: this migration number is (now) 29.

@rtldg rtldg merged commit 349ba58 into shavitush:master Oct 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants