Skip to content

Always send recording_id as part of LogMsg#1778

Merged
jleibs merged 9 commits intomainfrom
jleibs/send_recording_id
Apr 6, 2023
Merged

Always send recording_id as part of LogMsg#1778
jleibs merged 9 commits intomainfrom
jleibs/send_recording_id

Conversation

@jleibs
Copy link
Copy Markdown
Contributor

@jleibs jleibs commented Apr 5, 2023 •
edited
Loading

Our comms were previously somewhat stateful -- we assumed all messages received after a BeginRecording belonged to that recording. However, smart channels (as well as rrd files, web sockets, etc.) allow many sessions, including those with different recording_ids, to become interlaced, introducing ambiguity as to which recording a given message belonged to.

This adds some additional overhead to the LogMsg by sending RecordingId per-message.

Before:

Arrow payload containing a Pos2 uses 4493 bytes in RAM
Arrow LogMsg containing a Pos2 uses 10821-15418 bytes in RAM, and 805 bytes encoded
Arrow payload containing a Pos2 uses 27904 bytes in RAM
Arrow LogMsg containing 1000x Pos2 uses 34797-38829 bytes in RAM, and 8091 bytes encoded

After:

Arrow payload containing a Pos2 uses 4493 bytes in RAM
Arrow LogMsg containing a Pos2 uses 10821-15418 bytes in RAM, and 827 bytes encoded
Arrow payload containing a Pos2 uses 27904 bytes in RAM
Arrow LogMsg containing 1000x Pos2 uses 34797-38829 bytes in RAM, and 8121 bytes encoded

Performance benches seem to be in the noise:

group                                                 main                                     send_recording_id
-----                                                 ----                                     -----------------
batch_points_arrow/decode_log_msg                     1.00    198.1±4.83µs 48.1 MElem/sec      1.01    200.8±6.73µs 47.5 MElem/sec
batch_points_arrow/decode_message_bundles             1.01  1168.7±27.88ns  8.0 GElem/sec      1.00  1154.9±24.84ns  8.1 GElem/sec
batch_points_arrow/decode_total                       1.00    203.1±6.74µs 46.9 MElem/sec      1.00    203.3±8.41µs 46.9 MElem/sec
batch_points_arrow/encode_log_msg                     1.00    200.6±4.14µs 47.5 MElem/sec      1.07    215.2±3.51µs 44.3 MElem/sec
batch_points_arrow/encode_total                       1.00    367.8±6.64µs 25.9 MElem/sec      1.03   379.8±10.91µs 25.1 MElem/sec
batch_points_arrow/generate_message_bundles           1.01    154.3±2.79µs 61.8 MElem/sec      1.00    153.5±2.24µs 62.1 MElem/sec
batch_points_arrow/generate_messages                  1.00      3.0±0.06µs  3.1 GElem/sec      1.01      3.0±0.03µs  3.1 GElem/sec
mono_points_arrow/decode_log_msg                      1.01    108.3±1.07ms 90.1 KElem/sec      1.00    107.4±2.19ms 91.0 KElem/sec
mono_points_arrow/decode_message_bundles              1.03     30.2±0.41ms 322.9 KElem/sec     1.00     29.5±0.44ms 331.2 KElem/sec
mono_points_arrow/decode_total                        1.01    140.4±0.82ms 69.6 KElem/sec      1.00    139.3±1.51ms 70.1 KElem/sec
mono_points_arrow/encode_log_msg                      1.02     81.2±1.16ms 120.3 KElem/sec     1.00     79.5±0.90ms 122.8 KElem/sec
mono_points_arrow/encode_total                        1.01    167.1±1.07ms 58.4 KElem/sec      1.00    166.2±2.00ms 58.8 KElem/sec
mono_points_arrow/generate_message_bundles            1.03     22.3±1.80ms 438.7 KElem/sec     1.00     21.6±0.40ms 451.8 KElem/sec
mono_points_arrow/generate_messages                   1.00     65.4±0.76ms 149.2 KElem/sec     1.03     67.4±4.63ms 144.8 KElem/sec
mono_points_arrow_batched/decode_log_msg              1.03   441.0±18.96µs 21.6 MElem/sec      1.00   429.1±13.29µs 22.2 MElem/sec
mono_points_arrow_batched/decode_message_bundles      1.01      6.2±0.12ms 1582.3 KElem/sec    1.00      6.1±0.29ms 1600.5 KElem/sec
mono_points_arrow_batched/decode_total                1.03      6.9±0.17ms 1407.7 KElem/sec    1.00      6.8±0.17ms 1445.0 KElem/sec
mono_points_arrow_batched/encode_log_msg              1.01   780.2±11.06µs 12.2 MElem/sec      1.00   773.8±15.92µs 12.3 MElem/sec
mono_points_arrow_batched/encode_total                1.00     25.2±0.49ms 387.9 KElem/sec     1.01     25.4±1.41ms 385.1 KElem/sec
mono_points_arrow_batched/generate_message_bundles    1.04     19.0±0.31ms 515.0 KElem/sec     1.00     18.2±0.34ms 537.7 KElem/sec
mono_points_arrow_batched/generate_messages           1.05      4.9±0.17ms 1973.2 KElem/sec    1.00      4.7±0.19ms  2.0 MElem/sec

However, this overhead will be substantially mitigated by the new batching logic.

Resolves: #903

Checklist

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emilk emilk emilk approved these changes

+1 more reviewer

@teh-cmc teh-cmc teh-cmc left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

🪳 bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Log always goes to the active recording

3 participants

@jleibs @emilk @teh-cmc