3. Values
A JSON value MUST be an object, array, number, or string, or one of
the following three literal names:
false
null
true
The literal names MUST be lowercase. No other literal names are
allowed.
mysql> create table jsontest (id int not null auto_increment primary key, json_data json not null);
Query OK, 0 rows affected (0.03 sec)
mysql> insert into jsontest (json_data) values ('null');
Query OK, 1 row affected (0.00 sec)
mysql> insert into jsontest (json_data) values ('true');
Query OK, 1 row affected (0.01 sec)
mysql> insert into jsontest (json_data) values ('false');
Query OK, 1 row affected (0.01 sec)
mysql> insert into jsontest (json_data) values ('nil');
ERROR 3140 (22032): target: commerce.0.primary: vttablet: rpc error: code = Unknown desc = Invalid JSON text: "Invalid value." at position 1 in value for column 'jsontest.json_data'. (errno 3140) (sqlstate 22032) (CallerID: userData1): Sql: "insert into jsontest(json_data) values (:vtg1 /* VARCHAR */)", BindVars: {vtg1: "type:VARCHAR value:\"nil\""}
mysql> select * from jsontest;
+----+-----------+
| id | json_data |
+----+-----------+
| 1 | null |
| 2 | true |
| 3 | false |
+----+-----------+
3 rows in set (0.01 sec)
VReplication, however, in both the copy (vcopier) and running/replication (vplayer) phases turns the 'null' string literal into an SQL NULL value. This results in an incorrect copy of the data and if the column is NOT NULL then the workflow will fail.
git checkout main
make build
pushd examples/local
./101_initial_cluster.sh
vtctldclient ApplySchema --ddl-strategy=direct --sql='alter table corder add json_data json' commerce
mysql -e "insert into corder (customer_id, sku, price, json_data) values (1, 'sku1', 1, NULL), (2, 'sku2', 2, '{}'), (3, 'sku3', 3, 'null'), (4, 'sku4', 4, 'true'), (5, 'sku5', 5, 'false'), (10, 'sku10', 10, NULL)"
./201_customer_tablets.sh
./202_move_tables.sh
sleep 5
mysql -e "insert into corder (customer_id, sku, price, json_data) values (6, 'sku6', 6, 'true'), (7, 'sku7', 7, 'false'), (8, 'sku8', 8, 'null'), (9, 'sku9', 9, NULL)"
vtctldclient Workflow --keyspace=customer show --workflow=commerce2customer | jq '.workflows[0].shard_streams | .[].streams[0] | .state'
sleep 5
mysql customer:0 -e "select * from corder" --binary-as-hex=false
mysql commerce:0 -e "select * from corder" --binary-as-hex=false
vtctlclient vdiff -- --verbose --format=json customer.commerce2customer
vtctlclient vdiff -- --verbose --format=json customer.commerce2customer show last
./401_teardown.sh
popd
Overview of the Issue
According to the JSON spec: https://www.rfc-editor.org/rfc/rfc8259
And these values are handled correctly in vtgate:
VReplication, however, in both the copy (vcopier) and running/replication (vplayer) phases turns the 'null' string literal into an SQL
NULLvalue. This results in an incorrect copy of the data and if the column isNOT NULLthen the workflow will fail.Reproduction Steps
Binary Version
Version: 18.0.0-SNAPSHOT (Git revision f71583b6ef586faba07a2cf295831ccd47757fc8 branch 'main') built on Fri Sep 8 09:34:56 EDT 2023 by matt@pslord.local using go1.21.0 darwin/arm64Operating System and Environment details
Log Fragments