…s when the table has sequence column and add cases #25346" (#25789)

…s sequence column and add cases (apache#25346)

This PR fix the alignment process during publish phase when conflict occurs during concurrent partial updates: if we encounter a row with the same key and larger value in sequence column, it means that there exists another load which introduces a row with the same keys and larger sequence column value published successfully after the commit phase of the current load. We should act as follows:

- If the columns we update include sequence column, we should delete the current row becase the partial update on the current row has been overwritten by the previous one with larger sequence column value.
- Otherwise, we should combine the values of the missing columns in the previous row and the values of the including columns in the current row into a new row.

…s sequence column and add cases (apache#25346)

This PR fix the alignment process during publish phase when conflict occurs during concurrent partial updates: if we encounter a row with the same key and larger value in sequence column, it means that there exists another load which introduces a row with the same keys and larger sequence column value published successfully after the commit phase of the current load. We should act as follows:

- If the columns we update include sequence column, we should delete the current row becase the partial update on the current row has been overwritten by the previous one with larger sequence column value.
- Otherwise, we should combine the values of the missing columns in the previous row and the values of the including columns in the current row into a new row.

…update (#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert #25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.

…update (apache#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert apache#25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.

…update (apache#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert apache#25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.

…update (apache#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert apache#25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.

…update (apache#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert apache#25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.

…update (apache#41701)

1. Fix incorrect result when there are insert after delete for same key
in one load. In flexible partial update, if there is an insert after a
delete on the same key in one batch, the final result will be always
delete that row in table, which is incorrect. The reason is that when
doing aggregation for rows with same keys in memtable, we treat delete
sign column as an ordinary column. So if there is a row with delete sign
in that batch, the final aggregated row will always have a delete sign
and any insert on the same key after it will be lost.
In the above situation, we should keep those rows and don't merge them
when doing aggregation in memtable. When flush the memtable to segment,
we should first apply the semantic of delete if exists and then insert
the row after it.
2. fix flexible partial update don't use row store column to read data
3. Fix flexible partial update incorrect read index when resolving
conflict in publish phase
4. add some default value cases and fix on update current timestamp
5. fix publish phase alignment problem when table has sequence column
and the rowset to read has multi segments.
6. add some publish alignment case when table has sequence column
7. ~revert apache#25346, we use the
sequence value in flush phase as its final value and will not do
alignment in publish phase if the partial update load doesn't specified
sequence column.~
8. fix auto-increment column value is wrongly updated when it's not
specified.
9. Fix the problem that the value of sequence column may decrease.
10. Only read seq col and seq map col when the previous row is deleted
in partial update and flexible partial update when doing alignment.
11. Fix the value of auto-increment column is not consistent between
replicas.