Skip to content

Conversation

@zhiqiang-hhhh
Copy link
Contributor

Needed by #26278

@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In build.sh line 556:
    NPM='npm'
    ^-------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 557:
    if ! ${NPM} --version; then
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
         ^--------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 558:
        echo "Error: npm is not found"
        ^----------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 559:
        exit 1
        ^----^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 561:
    if [[ -n "${CUSTOM_NPM_REGISTRY}" ]]; then
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
       ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 562:
        "${NPM}" config set registry "${CUSTOM_NPM_REGISTRY}"
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 563:
        npm_reg="$("${NPM}" get registry)"
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
                   ^-------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 564:
        echo "NPM registry: ${npm_reg}"
        ^-----------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 567:
    echo "Build Frontend UI"
    ^----------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 568:
    ui_dist="${DORIS_HOME}/ui/dist"
    ^-----------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 569:
    if [[ -n "${CUSTOM_UI_DIST}" ]]; then
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).
       ^--------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 570:
        ui_dist="${CUSTOM_UI_DIST}"
        ^-------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 572:
        cd "${DORIS_HOME}/ui"
        ^-------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 573:
        "${NPM}" cache clean --force
        ^--------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 574:
        "${NPM}" install --legacy-peer-deps
        ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 575:
        "${NPM}" run build
        ^----------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 577:
    echo "ui dist: ${ui_dist}"
    ^------------------------^ SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 578:
    rm -rf "${DORIS_HOME}/fe/fe-core/src/main/resources/static"
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 579:
    mkdir -p "${DORIS_HOME}/fe/fe-core/src/main/resources/static"
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).


In build.sh line 580:
    cp -r "${ui_dist}"/* "${DORIS_HOME}/fe/fe-core/src/main/resources/static"/
    ^-- SC2317 (info): Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

For more information:
  https://www.shellcheck.net/wiki/SC2317 -- Command appears to be unreachable...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- build.sh.orig
+++ build.sh
@@ -537,9 +537,9 @@
         -DDORIS_JAVA_HOME="${JAVA_HOME}" \
         "${DORIS_HOME}/be"
 
-        "${BUILD_SYSTEM}" -j "${PARALLEL}"    
-        # 不要 install,这样确保 workspace 的 debug 跟 release 可以使用软连接到不同的二进制
-        # "${BUILD_SYSTEM}" install
+    "${BUILD_SYSTEM}" -j "${PARALLEL}"
+    # 不要 install,这样确保 workspace 的 debug 跟 release 可以使用软连接到不同的二进制
+    # "${BUILD_SYSTEM}" install
 
     cd "${DORIS_HOME}"
 fi
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


@zhiqiang-hhhh zhiqiang-hhhh marked this pull request as ready for review November 24, 2023 03:22
@zhiqiang-hhhh
Copy link
Contributor Author

run buildall

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit b7f6b3e4432cb0f144579924956ee88f54817b9b, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4891	4633	4655	4633
q2	370	144	179	144
q3	2044	1930	1883	1883
q4	1367	1262	1253	1253
q5	3942	3911	3998	3911
q6	250	129	130	129
q7	1437	893	876	876
q8	2762	2780	2741	2741
q9	58312	13119	9384	9384
q10	13093	3510	3508	3508
q11	395	247	251	247
q12	1633	293	294	293
q13	21391	3762	3776	3762
q14	312	270	288	270
q15	579	535	525	525
q16	671	597	578	578
q17	1144	955	895	895
q18	7704	7343	7345	7343
q19	1714	1666	1653	1653
q20	578	324	289	289
q21	7459	4011	3972	3972
q22	472	378	374	374
Total cold run time: 132520 ms
Total hot run time: 48663 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4646	4573	4578	4573
q2	337	254	251	251
q3	4019	4001	3981	3981
q4	2704	2686	2690	2686
q5	9665	9549	9651	9549
q6	245	125	126	125
q7	3015	2484	2505	2484
q8	4405	4448	4394	4394
q9	12856	12885	12855	12855
q10	4073	4134	4152	4134
q11	814	704	637	637
q12	973	825	813	813
q13	4261	3577	3545	3545
q14	385	361	351	351
q15	576	521	519	519
q16	721	659	666	659
q17	3949	3920	3902	3902
q18	9581	9065	9078	9065
q19	1830	1781	1790	1781
q20	2400	2051	2043	2043
q21	8707	8401	8430	8401
q22	930	766	778	766
Total cold run time: 81092 ms
Total hot run time: 77514 ms

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.45 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.2 seconds inserted 10000000 Rows, about 342K ops/s
storage size: 17099123323 Bytes

Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei yiguolei merged commit e4258dd into apache:master Nov 24, 2023
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 24, 2023
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@zhiqiang-hhhh zhiqiang-hhhh deleted the chore-plugin branch November 24, 2023 11:32
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Nov 27, 2023
eldenmoon added a commit that referenced this pull request Nov 27, 2023
* [fix](stats) Fix update rows for unique table didn't get updated properly #26968 (#27337)

* [FIX](jsonb) fix jsonb in predict column #27325 (#27424)

* [fix](fe) slots in having clause should be set to need materialized(#27412) (#27429)

* [Bug](insert)fix insert wrong data on mv when stmt have multiple values (#27297) (#27382)

fix insert wrong data on mv when stmt have multiple values

* [fix](fe ut) Fix OlapQueryCacheTest failed (#27305) (#27406)

1.
```
java.lang.NullPointerException: null
        at org.apache.doris.catalog.Env.getCurrentSystemInfo(Env.java:793) ~[classes/:?]
        at org.apache.doris.qe.SimpleScheduler$UpdateBlacklistThread.run(SimpleScheduler.java:206) ~[classes/:?]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]

java.lang.NullPointerException
        at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:226)
```

2.
```
[ERROR] testSqlCacheKeyWithNestedViewForNereids  Time elapsed: 1.962 s  <<< FAILURE!
java.lang.AssertionError: SELECT command denied to user 'testCluster:testUser'@'192.168.1.1' for table 'internal: testCluster:testDb: appevent'
	at org.apache.doris.qe.OlapQueryCacheTest.parseSqlByNereids(OlapQueryCacheTest.java:579)
	at org.apache.doris.qe.OlapQueryCacheTest.testSqlCacheKeyWithNestedViewForNereids(OlapQueryCacheTest.java:1338)
```

3.
```
[ERROR] Tests run: 28, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 113.63 s <<< FAILURE! - in org.apache.doris.qe.OlapQueryCacheTest
[ERROR] testCacheModeTable  Time elapsed: 1.657 s  <<< ERROR!
java.lang.IllegalArgumentException: Value of type org.apache.doris.qe.QueryState incompatible with return type org.apache.doris.system.SystemInfoService of org.apache.doris.catalog.Env#getCurrentSystemInfo()
        at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:156)
```

* [regression test](schema change) add some schema change regression cases (#27112) (#27418)

* [fix](Nereids) result type of add precision is 1 more than expected (#27136) (#27426)

* [fix](Nereids): fill miss slot in having subquery (#27177) (#27394)

* [fix](memory) Fix make_top_consumption_snapshots heap-use-after-free #27434 (#27465)

* [fix](function) make TIMESTAMP function DEPEND_ON_ARGUMENT (#27343) (#27458)

* [fix](test) order by clause in test_map(#27390) (#27391)

pick #27390

* [performance](Planner): optimize getStringValue() in DateLiteral (#27363) (#27470)

- reduce cost of `getStringValue()`
- original code don't consider `microsecond` part in `getStringValue()`

(cherry picked from commit 044a295)

* [Chore](pick) do not push down agg on aggregate column (#27356) (#27498)

* [fix](stats) table not exists error msg not print objects name #27074 (#27463)

* [improve](nereids) support agg function of count(const value) pushdown #26677 (#27499)

support sql: select count(1)-count(not null) from table, the agg of count could push down.

* [test](fe-ut) fix unstable MysqlServerTest (#27459)

Need to find a unbind port for MysqlServerTest

* [opt](MergedIO) no need to merge large columns (#27315) (#27497)

1. Fix a profile bug of `MergeRangeFileReader`, and add a profile `ApplyBytes` to show the total bytes  of ranges.
2. There's no need to merge large columns, because `MergeRangeFileReader` will increase the copy time.

* [improvement](drop tablet)  impr gc shutdown tablet lock (#26151) (#27478)

* [doc](stats) SQL manual for stats (#27461)

* [chore](merge-on-write) disable rowid conversion check for mow table by default (#27482) (#27508)

* [fix](regression)Fix hive p2 case (#27466) (#27511)

* [fix](statistics)Fix auto analyze remove finished job bug #27486 (#27510)

* [Bug](bitmap) Fix heap-use-after-free in the bitmap functions #27411 (#27521)

* [Pick](nereids) Pick: partition prune fails in case of NOT expression (#27047) (#27507)

* [fix](clone) Fix engine_clone file exist (#27361) (#27536)

* [chore](case) adjust timeout of broker load case #27540

* Fix auto analyze doesn't filter unsupported type bug. (#27547)

Fix auto analyze doesn't filter unsupported type bug.
Catch throwable in auto analyze thread for each database, otherwise the thread will quit when one database failed to create jobs and all other databases will not get analyzed.
change FE config item full_auto_analyze_simultaneously_running_task_num to auto_analyze_simultaneously_running_task_num
backport #27559

* [chore](fe plugin) Upgrade dependency to doris 2.0-SNAPSHOT #27522 (#27558)

* [Bug](materialized-view) add limitation for duplicate expr on materialized view (#27523) (#27562)

* [fix](planner)join node should output required slot from parent node #27526 (#27551)

* [branch-2.0](hive) enable hive view by default (#27550)

* [pick](nereids) adjust bc join and shuffle join #27113 (#27566)

* [Fix](hive-transactional-table) Fix NPE when query empty hive transactional table. (#27567)

---------

Co-authored-by: AKIRA <33112463+Kikyou1997@users.noreply.github.com>
Co-authored-by: amory <wangqiannan@selectdb.com>
Co-authored-by: Jerry Hu <mrhhsg@gmail.com>
Co-authored-by: Pxl <pxl290@qq.com>
Co-authored-by: Xinyi Zou <zouxinyi02@gmail.com>
Co-authored-by: Luwei <814383175@qq.com>
Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com>
Co-authored-by: 谢健 <jianxie0@gmail.com>
Co-authored-by: Mryange <59914473+Mryange@users.noreply.github.com>
Co-authored-by: jakevin <jakevingoo@gmail.com>
Co-authored-by: zhangstar333 <87313068+zhangstar333@users.noreply.github.com>
Co-authored-by: Mingyu Chen <morningman@163.com>
Co-authored-by: Ashin Gau <AshinGau@users.noreply.github.com>
Co-authored-by: yujun <yu.jun.reach@gmail.com>
Co-authored-by: Xin Liao <liaoxinbit@126.com>
Co-authored-by: Jibing-Li <64681310+Jibing-Li@users.noreply.github.com>
Co-authored-by: xy720 <22125576+xy720@users.noreply.github.com>
Co-authored-by: minghong <englefly@gmail.com>
Co-authored-by: Jack Drogon <jack.xsuperman@gmail.com>
Co-authored-by: Dongyang Li <hello_stephen@qq.com>
Co-authored-by: zhiqiang <seuhezhiqiang@163.com>
Co-authored-by: starocean999 <40539150+starocean999@users.noreply.github.com>
Co-authored-by: Qi Chen <kaka11.chen@gmail.com>
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
gnehil pushed a commit to gnehil/doris that referenced this pull request Dec 4, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.3-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants