Skip to content

Bug Report: VReplication does not handle 'null' JSON literal values correctly #13942

@mattlord

Description

@mattlord

Overview of the Issue

According to the JSON spec: https://www.rfc-editor.org/rfc/rfc8259

3.  Values

   A JSON value MUST be an object, array, number, or string, or one of
   the following three literal names:

      false
      null
      true

   The literal names MUST be lowercase.  No other literal names are
   allowed.

And these values are handled correctly in vtgate:

mysql> create table jsontest (id int not null auto_increment primary key, json_data json not null);
Query OK, 0 rows affected (0.03 sec)

mysql> insert into jsontest (json_data) values ('null');
Query OK, 1 row affected (0.00 sec)

mysql> insert into jsontest (json_data) values ('true');
Query OK, 1 row affected (0.01 sec)

mysql> insert into jsontest (json_data) values ('false');
Query OK, 1 row affected (0.01 sec)

mysql> insert into jsontest (json_data) values ('nil');
ERROR 3140 (22032): target: commerce.0.primary: vttablet: rpc error: code = Unknown desc = Invalid JSON text: "Invalid value." at position 1 in value for column 'jsontest.json_data'. (errno 3140) (sqlstate 22032) (CallerID: userData1): Sql: "insert into jsontest(json_data) values (:vtg1 /* VARCHAR */)", BindVars: {vtg1: "type:VARCHAR value:\"nil\""}

mysql> select * from jsontest;
+----+-----------+
| id | json_data |
+----+-----------+
|  1 | null      |
|  2 | true      |
|  3 | false     |
+----+-----------+
3 rows in set (0.01 sec)

VReplication, however, in both the copy (vcopier) and running/replication (vplayer) phases turns the 'null' string literal into an SQL NULL value. This results in an incorrect copy of the data and if the column is NOT NULL then the workflow will fail.

Reproduction Steps

git checkout main

make build

pushd examples/local

./101_initial_cluster.sh

vtctldclient ApplySchema --ddl-strategy=direct --sql='alter table corder add json_data json' commerce

mysql -e "insert into corder (customer_id, sku, price, json_data) values (1, 'sku1', 1, NULL), (2, 'sku2', 2, '{}'), (3, 'sku3', 3, 'null'), (4, 'sku4', 4, 'true'), (5, 'sku5', 5, 'false'), (10, 'sku10', 10, NULL)"

./201_customer_tablets.sh

./202_move_tables.sh

sleep 5

mysql -e "insert into corder (customer_id, sku, price, json_data) values (6, 'sku6', 6, 'true'), (7, 'sku7', 7, 'false'), (8, 'sku8', 8, 'null'), (9, 'sku9', 9, NULL)"

vtctldclient Workflow --keyspace=customer show --workflow=commerce2customer | jq '.workflows[0].shard_streams | .[].streams[0] | .state'

sleep 5

mysql customer:0 -e "select * from corder" --binary-as-hex=false

mysql commerce:0 -e "select * from corder" --binary-as-hex=false

vtctlclient vdiff -- --verbose --format=json customer.commerce2customer

vtctlclient vdiff -- --verbose --format=json customer.commerce2customer show last

./401_teardown.sh

popd

Binary Version

Version: 18.0.0-SNAPSHOT (Git revision f71583b6ef586faba07a2cf295831ccd47757fc8 branch 'main') built on Fri Sep  8 09:34:56 EDT 2023 by matt@pslord.local using go1.21.0 darwin/arm64

Operating System and Environment details

N/A

Log Fragments

N/A

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions