Get rid of pthread_cancel() #8423
Description
Motivation:
- In general, thread cancellation is not a good idea, it is better to do a graceful shutdown.
pthread_cancel()doesn't work well in macOS:
flaky test:gh_3211_per_module_log_level_test.luaon macOS #8420 (comment)
See also test: flaky issue that replicas can't be terminated with SIGTERM on OSX tarantool-qa#32 (comment)
Currently pthread_cancel() is used by:
src/box/applier.cc:2106: cord_cancel_and_join(&thread->cord);
src/box/iproto.cc:3562: cord_cancel_and_join(&iproto_threads[i].net_cord);
src/box/memtx_engine.cc:836: cord_cancel_and_join(&ckpt->cord);
src/box/memtx_engine.cc:848: cord_cancel_and_join(replica_join_cord);
src/box/relay.cc:356: cord_cancel_and_join(&relay->cord);
src/box/vy_run.c:169: cord_cancel_and_join(&reader->cord);
src/box/vy_scheduler.c:359: cord_cancel_and_join(&worker->cord);