Skip to content

Increase XLOG_READ_AHEAD #8108

New issue
New issue
Closed
#8113
Closed
Increase XLOG_READ_AHEAD#8108
#8113
Assignees
Gumix
Labels
2.11Target is 2.11 and all newer release/master branchesfeatureA new functionalityperformance
@Gumix

Description

@Gumix
Gumix
opened on Dec 29, 2022

xlog and snap files are compressed, xlog_cursor_next reads them from HDD by ~50 KB chunks:

tarantool/src/box/xlog.c

Lines 1898 to 1902 in 182034d

while ((to_load = xlog_tx_cursor_create(&i->tx_cursor,
(const char **)&i->rbuf.rpos,
i->rbuf.wpos, i->zdctx)) > 0) {
/* not enough data in read buffer */
int rc = xlog_cursor_ensure(i, ibuf_used(&i->rbuf) + to_load);

tarantool/src/box/xlog.c

Line 1548 in 182034d

to_load += XLOG_READ_AHEAD;

However XLOG_READ_AHEAD is only 16 KB, which doesn't make any sense.

tarantool/src/box/xlog.c

Lines 1557 to 1558 in 182034d

readen = fio_pread(cursor->fd, dst, to_load,
cursor->read_offset);

Here are CPU usage graphs for Tarantool recovery from a 900 MB snapshot (see #5787 "trees with hints") from a HDD with 300 MB/sec read speed (buffered, uncached).

The first run (# sync; echo 1 > /proc/sys/vm/drop_caches) took 25 sec:

Screenshot 2022-12-29 at 21 26 43

The second run (with hot page cache) took 12 sec:

Screenshot 2022-12-29 at 21 27 26

The first run is obviously I/O bound, it should be up to 2x faster with a big enough XLOG_READ_AHEAD.

Metadata

Metadata

Assignees

Labels

2.11Target is 2.11 and all newer release/master branchesfeatureA new functionalityperformance

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions