libct/system.Stat: fix panic, speed up, add benchmarks#2696
libct/system.Stat: fix panic, speed up, add benchmarks#2696cyphar merged 3 commits intoopencontainers:masterfrom
Conversation
|
Rebased to benefit from #2690 (while Travis was not able to test this PR for 6+ hours 😢) |
|
@AkihiroSuda @mrunalp PTAL |
|
rebased (for new CI) |
thaJeztah
left a comment
There was a problem hiding this comment.
LGTM, but left some thoughts/suggestions for consideration
|
Is this a bugfix or an improvement? If it's an improvement it belongs in post-1.0 as a milestone. |
Improvement (except eliminating a linter warning which might be classified as bugfix), and that's lots of code, so moving to post 1.0 |
1ca7e5c to
8d6a5d4
Compare
|
Yeah agreed, this change looks a little scary tbh. |
|
close/open to re-kick ci |
|
v3: rebased; drop using pkg/errors; gofumpt'ed |
|
CI failure is unrelated (#3050) |
|
Rebased to fix CI. |
f4716d3 to
89deda8
Compare
|
Some quite interesting observations:
|
|
@thaJeztah PTAL |
thaJeztah
left a comment
There was a problem hiding this comment.
overall looks good; I was looking for corner-cases, and possibly found some (may not be important to add, but thought I'd post them)
I'm fine with merging as-is though
1. Add a test case that tests parentheses in command. 2. Replace individual comparisons with reflect.DeepEqual. This also fixes wrong %-style types in Fatalf statements. 3. Replace Fatalf with Errorf so we don't bail out on the first failure, and do not check result on error. 4. Add two benchmarks. On my laptop, they show: BenchmarkParseStat-4 116415 10804 ns/op BenchmarkParseRealStat-4 240 4781769 ns/op Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Remove PID field as it is useless. 2. Rewrite parseStat() to make it faster and more correct: - do not use fmt.Scanf as it is very slow; - avoid splitting data into 20+ fields, of which we only need 2; - make sure to not panic on short lines and other bad input; - add some bad input tests (some fail with old code); - use LastIndexByte instead of LastIndex. Benchmarks: before (from the previous commit message): > BenchmarkParseStat-4 116415 10804 ns/op > BenchmarkParseRealStat-4 240 4781769 ns/op after: > BenchmarkParseStat-4 1164948 1068 ns/op > BenchmarkParseRealStat-4 331 3458315 ns/op We are seeing 10x speedup in a synthetic benchmark, and about 1.4x speedup in a real world benchmark. While at it, do not ignore any possible errors, and properly wrap those. [v2: use pkg/errors more, remove t.Logf from test] [v3: rebased; drop pkg/errors; gofumpt'ed] [v4: rebased; improved description] [v5: rebased; mention bad input tests, added second benchmark results] [v6: remove PID field, do not use strings.Split, further speedup] Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Those states are available since Linux 4.14 (kernel commits 8ef9925b02c23e3838d5 and 06eb61844d841d003). Before this patch, they were shown as unknown. This is mostly cosmetical. Note that I is described in /proc/pid/status as just "idle", although elsewhere it says it's an idle kernel thread. Let's have it as "idle" for brevity. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
| // 1. Look for the first '(' and the last ')' first, what's in between is Name. | ||
| // We expect at least 20 fields and a space after the last one. |
There was a problem hiding this comment.
Here's hoping that they don't add a new field in a few kernel versions that contains ) grimace. Why on earth doesn't /proc/self/status contain StartTime? If it did we could just use that rather than this unparseable crap. /rant
There was a problem hiding this comment.
I heard they will switch to YAML, because it seems to be popular 🤪
1. libct/system/proc_test: fix, improve, add benchmarks
Add a test case that tests parentheses in command.
Replace individual comparisons with
reflect.DeepEqual.This also fixes wrong
%-style types used inFatalfstatements.Replace
FatalfwithErrorfso we don't bail out on the firstfailure, and do not check result on error.
Add two benchmarks.
2. libct/system.Stat: fix/improve/speedup
Remove PID field. It is totally useless -- we already know the PID.
Rewrite
parseStat()to make it faster and more correct:fmt.Scanfas it is very slow;LastIndexByteinstead ofLastIndexwhere appropriate.Benchmarks:
before (from the previous commit message):
after:
We are seeing 10x speedup in a synthetic benchmark, and about 1.4x
speedup in a real world benchmark. All this while being more scrupulous
about any possible errors. I also suspect much less allocations and thus
garbage to collect.
While at it, do not ignore any possible errors, and properly wrap those.
IandPprocess states (available since Linux 4.14).History