Overview

• Proposal: Improve the performance of JavaScript version of OpenCV (OpenCV.js)
• Project Report
• Mentor: Ningxin Hu @huningxin , Vitaly Tuzov @terfendail
• Student: Wenzhao Xiang @Wenzhao-Xiang

This pullrequest changes

• Create the base of OpenCV.js performance test:
  This perf test is based on benchmark.js. And we first add cvtColor, Resize, Threshold into it. We support both browser and Node.js version of it for test.
• Optimize the OpenCV.js performance by WASM threads:
  This optimization is based on Web Worker API and SharedArrayBuffer, so it can be only used in browser. We expose two new API cv.parallel_pthreads_set_threads_num(number) and cv.parallel_pthreads_get_threads_num(), so we can use the former to set threads number dynamically and use the latter to get the current threads number. And the default threads number is the logic core number of the device.
• Optimize the OpenCV.js performance by WASM SIMD:
  Add WASM SIMD backend for OpenCV Universal Intrinsics. It's experimental as WASM SIMD is still in development. The simd version of OpenCV.js built by latest LLVM upstream may not work with the stable browser or old version of Node.js. Please use the latest version of unstable browser or Node.js to get new features, like Chrome Dev.

The Test

Test Environment:

  OS: Ubuntu 16.04
  Emscripten: 1.38.42, LLVM upstream backend
  Browser: Chrome, Version 78.0.3880.4 (Official Build) dev (64-bit)
  Hardware: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz with ８ logical cores:

• OpenCV.js tests: all passed
• Universal Intrinsics WASM backend test: all passed

Results

• For OpenCV kernels, take Threshold kernel with parameter (1920x1080, CV_8UC1, THRESH_BINARY) as example:

OS: Ubuntu 16.04<br>
Emscripten: 1.38.42, LLVM upstream backend<br>
Browser: Chrome, Version 78.0.3880.4 (Official Build) dev (64-bit)<br>
Hardware: Core(TM) i7-8700 CPU @ 3.20GHz with 12 logical cores

• For real case, take OpenCV.js face recognition sample as example:

OS: Ubuntu Linux 16.04.5
Emscripten: 1.38.42, LLVM upstream backend
Browser: Chrome, Version 78.0.3880.4 (Official Build) dev (64-bit)
Hardware: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz with 12 logical cores

• The OpenCV.js setup tutorial
• The OpenCV.js demos (May need the latest version of Chrome-Dev)
• GSoC report video

Performance Analysis

Kernel performance(ms)

Test Environment:
OS: Ubuntu 16.04
Emscripten: 1.38.42, LLVM upstream backend
Browser: Chrome, Version 78.0.3880.4 (Official Build) dev (64-bit)
Hardware: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz with 8 logical cores:

Analysis

With the current optimization, threads optimization works as we expected. However, wasm simd still have some issues. As we can see in the Kernel performance result, now resize only have 1.34x speed up than scalar version and cvtColor is even 2-3x slower than scalar version, which still have a big gap compared with Native SIMD optimization.
Thanks @huningxin for the investigation, here are some analysis results:

• For resize, the biggest reason is that we now use shift to simulation integer widening instructions in v_dotprod. We have opened an emscripten issue. And we can continue to optimize resize kernel after this new feature is enabled.
• For cvtColor, the root cause is the inefficient pshufb with memory operands are generated by V8 for current implementation.
  One solution is to refer to sse implementation that uses punpcklbw and punpckhqdq. We tried but it still fails due to an emscripten issue that leads V8 fails to generate those instructions. Let's see the response from emscripten community.

force_builders=Custom
buildworker:Custom=linux-1,linux-2,linux-4
build_image:Docs=docs-js
build_image:Custom=javascript