Skip to content

insert into语句性能问题 #5655

@htyoung

Description

@htyoung

集群配置:
4台16Core 64G内存机器,其中1台FE(硬盘500G SSD),3台BE(硬盘1T SSD)(一台混部FE Observer,混部FE单独500G SSD磁盘)

建表语句如下:
CREATE TABLE IF NOT EXISTS test_table_2
(
dateline BIGINT COMMENT '服务器时间,毫秒时间戳'
,dt date COMMENT '分区字段,天分区,yyyyMMdd'
,ht int COMMENT '分区字段,小时分区,格式HH'
)
DUPLICATE KEY(dateline,dt,ht)
COMMENT ''
PARTITION BY RANGE(dt, ht)
(
partition p20210401 VALUES [("2021-04-01","0"),("2021-04-01","24")),
partition p20210402 VALUES [("2021-04-02","0"),("2021-04-02","24")),
partition p20210403 VALUES [("2021-04-03","0"),("2021-04-03","24")),
partition p20210404 VALUES [("2021-04-04","0"),("2021-04-04","24")),
partition p20210405 VALUES [("2021-04-05","0"),("2021-04-05","24")),
partition p20210406 VALUES [("2021-04-06","0"),("2021-04-06","24")),
partition p20210407 VALUES [("2021-04-07","0"),("2021-04-07","24")),
partition p20210408 VALUES [("2021-04-08","0"),("2021-04-08","24")),
partition p20210409 VALUES [("2021-04-09","0"),("2021-04-09","24")),
partition p20210410 VALUES [("2021-04-10","0"),("2021-04-10","24")),
partition p20210411 VALUES [("2021-04-11","0"),("2021-04-11","24"))
)
DISTRIBUTED BY HASH(dateline) BUCKETS 16
;

插入语句:
insert into dy_dwd.dwd_dy_log_dot_hi_2(dateline,dt,ht) values(1618408080,'2021-04-01',5),(1618408081,'2021-04-01',6),......;
values中数据达到100条甚至更多

结果报错:
当values中数据达到100或者1000条甚至更多时,走mysql jdbc协议会出现如下错误
ERROR 5025 (HY000): errCode = 2, detailMessage = already stopped, skip waiting for close. cancelled/!eos: : 1/0

测试中发现每秒insert into的values个数据达到20之后就会出现上述错误,既然兼容MySQL协议,明显insert into导入数据的方式性能太低

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions