Skip to content

Commit f99b0cf

Browse files
github-actions[bot]kei-nan
github-actions[bot]
and
kei-nan
authored
[2.10] MOD-7745: Add Benchmarks For Document And Field Expiration (#5063)
MOD-7745: Add Benchmarks For Document And Field Expiration (#5030) * * initial commit * * rename internal names * * rename numeric field benchmark * * use second resolution to avoid actually expiring stuff and ruining our dataset * * typo * * add key limitations to benchmarks * * add two tests that will actually expire fields and documents * will probably be less deterministic but we want to see how it actually behaves * * change to a dataset where the __key__ can be effective * * fix benchmark name * * fix benchmark failures * * remove requests limitation * * updated lazy expiration benchmarks description (cherry picked from commit 3aeb659) Co-authored-by: kei-nan <jonathan.keinan@redis.com>
1 parent c4a9c73 commit f99b0cf

4 files changed

Lines changed: 146 additions & 0 deletions

‎tests/benchmarks/search-expire-doc-10-milliseconds.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
version: 0.2
2+
name: "search-expire-doc-10-milliseconds"
3+
description: "
4+
We want to check our ttl table performance, edge cases we need to consider when modifying expiration tests:
5+
- we want to keep an existing dataset, if we actually expire documents during the test we will decrease our keyspace size
6+
- if we reach a point where every single key is set to be expired then there will be a hot path which might not represent real world usage
7+
This specific benchmark focuses on lazy expiration by disabling active expiration
8+
"
9+
10+
metadata:
11+
component: "search"
12+
setups:
13+
- oss-standalone
14+
- oss-cluster-02-primaries
15+
- oss-cluster-04-primaries
16+
- oss-cluster-08-primaries
17+
- oss-cluster-16-primaries
18+
- oss-cluster-20-primaries
19+
- oss-cluster-24-primaries
20+
- oss-cluster-32-primaries
21+
22+
dbconfig:
23+
- tool: ftsb_redisearch
24+
- parameters:
25+
- workers: 64
26+
- reporting-period: 1s
27+
- input: "https://s3.amazonaws.com/benchmarks.redislabs/redisearch/datasets/5200K-docs-union-iterators/5200K-docs-union-iterators.idx10.commands.SETUP.csv"
28+
- init_commands:
29+
- '"DEBUG SET-ACTIVE-EXPIRE 0"'
30+
- '"FT.CREATE" "idx10" "PREFIX" "1" "idx10:" "SCHEMA" "field1" "NUMERIC" "field2" "TEXT" "field3" "TEXT" "field4" "NUMERIC" "field5" "NUMERIC" "field6" "NUMERIC" "field7" "NUMERIC" "field8" "NUMERIC" "field9" "NUMERIC"'
31+
- dataset_load_timeout_secs: 180
32+
- check:
33+
keyspacelen: 5204050
34+
35+
clientconfig:
36+
benchmark_type: "read-only"
37+
tool: memtier_benchmark
38+
arguments: "--test-time 180 -c 4 -t 1 --hide-histogram --key-prefix 'idx10:' --key-minimum 1 --key-maximum 100000 --command 'FT.SEARCH idx10 \"(( @field3: (1) @field6: [-inf 76.761 ] @field7: [76.761 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] )| ( @field3: (2) @field6: [-inf 137.894 ] @field7: [137.894 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] ))\"' --command-ratio 95 --command 'PEXPIRE __key__ 10' --command-ratio 5"

‎tests/benchmarks/search-expire-doc-1000-seconds.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
version: 0.2
2+
name: "search-expire-doc-1000-seconds"
3+
description: "
4+
We want to check our ttl table performance, edge cases we need to consider when modifying expiration tests:
5+
- we want to keep an existing dataset, if we actually expire documents during the test we will decrease our keyspace size
6+
- if we reach a point where every single key is set to be expired then there will be a hot path which might not represent real world usage
7+
"
8+
9+
metadata:
10+
component: "search"
11+
setups:
12+
- oss-standalone
13+
- oss-cluster-02-primaries
14+
- oss-cluster-04-primaries
15+
- oss-cluster-08-primaries
16+
- oss-cluster-16-primaries
17+
- oss-cluster-20-primaries
18+
- oss-cluster-24-primaries
19+
- oss-cluster-32-primaries
20+
21+
dbconfig:
22+
- tool: ftsb_redisearch
23+
- parameters:
24+
- workers: 64
25+
- reporting-period: 1s
26+
- input: "https://s3.amazonaws.com/benchmarks.redislabs/redisearch/datasets/5200K-docs-union-iterators/5200K-docs-union-iterators.idx10.commands.SETUP.csv"
27+
- init_commands:
28+
- '"FT.CREATE" "idx10" "PREFIX" "1" "idx10:" "SCHEMA" "field1" "NUMERIC" "field2" "TEXT" "field3" "TEXT" "field4" "NUMERIC" "field5" "NUMERIC" "field6" "NUMERIC" "field7" "NUMERIC" "field8" "NUMERIC" "field9" "NUMERIC"'
29+
- dataset_load_timeout_secs: 180
30+
- check:
31+
keyspacelen: 5204050
32+
33+
clientconfig:
34+
benchmark_type: "read-only"
35+
tool: memtier_benchmark
36+
arguments: "--test-time 180 -c 4 -t 1 --hide-histogram --key-prefix 'idx10:' --key-minimum 1 --key-maximum 100000 --command 'FT.SEARCH idx10 \"(( @field3: (1) @field6: [-inf 76.761 ] @field7: [76.761 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] )| ( @field3: (2) @field6: [-inf 137.894 ] @field7: [137.894 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] ))\"' --command-ratio 90 --command 'EXPIRE __key__ 1000' --command-ratio 10"

‎tests/benchmarks/search-expire-numeric-field-10-milliseconds.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
version: 0.2
2+
name: "search-expired-numeric-field-10-milliseconds"
3+
description: "
4+
We want to check our ttl table performance, edge cases we need to consider when modifying expiration tests:
5+
- we want to keep an existing dataset, if we actually expire documents during the test we will decrease our keyspace size
6+
- if we reach a point where every single field is set to be expired then there will be a hot path which might not represent real world usage
7+
This specific benchmark focuses on lazy expiration by disabling active expiration
8+
"
9+
metadata:
10+
component: "search"
11+
setups:
12+
- oss-standalone
13+
- oss-cluster-02-primaries
14+
- oss-cluster-04-primaries
15+
- oss-cluster-08-primaries
16+
- oss-cluster-16-primaries
17+
- oss-cluster-20-primaries
18+
- oss-cluster-24-primaries
19+
- oss-cluster-32-primaries
20+
21+
dbconfig:
22+
- tool: ftsb_redisearch
23+
- parameters:
24+
- workers: 64
25+
- reporting-period: 1s
26+
- input: "https://s3.amazonaws.com/benchmarks.redislabs/redisearch/datasets/5200K-docs-union-iterators/5200K-docs-union-iterators.idx10.commands.SETUP.csv"
27+
- init_commands:
28+
- '"DEBUG SET-ACTIVE-EXPIRE 0"'
29+
- '"FT.CREATE" "idx10" "PREFIX" "1" "idx10:" "SCHEMA" "field1" "NUMERIC" "field2" "TEXT" "field3" "TEXT" "field4" "NUMERIC" "field5" "NUMERIC" "field6" "NUMERIC" "field7" "NUMERIC" "field8" "NUMERIC" "field9" "NUMERIC"'
30+
- dataset_load_timeout_secs: 180
31+
- check:
32+
keyspacelen: 5204050
33+
34+
clientconfig:
35+
benchmark_type: "read-only"
36+
tool: memtier_benchmark
37+
arguments: "--test-time 180 -c 4 -t 1 --hide-histogram --key-prefix 'idx10:' --key-minimum 1 --key-maximum 100000 --command 'FT.SEARCH idx10 \"(( @field3: (1) @field6: [-inf 76.761 ] @field7: [76.761 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] )| ( @field3: (2) @field6: [-inf 137.894 ] @field7: [137.894 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] ))\"' --command-ratio 95 --command 'HPEXPIRE __key__ 10 FIELDS 2 field6 field3' --command-ratio 5"

‎tests/benchmarks/search-expire-numeric-field-1000-seconds.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
version: 0.2
2+
name: "search-expired-numeric-field-1000-seconds"
3+
description: "
4+
We want to check our ttl table performance, edge cases we need to consider when modifying expiration tests:
5+
- we want to keep an existing dataset, if we actually expire documents during the test we will decrease our keyspace size
6+
- if we reach a point where every single field is set to be expired then there will be a hot path which might not represent real world usage
7+
"
8+
metadata:
9+
component: "search"
10+
setups:
11+
- oss-standalone
12+
- oss-cluster-02-primaries
13+
- oss-cluster-04-primaries
14+
- oss-cluster-08-primaries
15+
- oss-cluster-16-primaries
16+
- oss-cluster-20-primaries
17+
- oss-cluster-24-primaries
18+
- oss-cluster-32-primaries
19+
20+
dbconfig:
21+
- tool: ftsb_redisearch
22+
- parameters:
23+
- workers: 64
24+
- reporting-period: 1s
25+
- input: "https://s3.amazonaws.com/benchmarks.redislabs/redisearch/datasets/5200K-docs-union-iterators/5200K-docs-union-iterators.idx10.commands.SETUP.csv"
26+
- init_commands:
27+
- '"FT.CREATE" "idx10" "PREFIX" "1" "idx10:" "SCHEMA" "field1" "NUMERIC" "field2" "TEXT" "field3" "TEXT" "field4" "NUMERIC" "field5" "NUMERIC" "field6" "NUMERIC" "field7" "NUMERIC" "field8" "NUMERIC" "field9" "NUMERIC"'
28+
- dataset_load_timeout_secs: 180
29+
- check:
30+
keyspacelen: 5204050
31+
32+
clientconfig:
33+
benchmark_type: "read-only"
34+
tool: memtier_benchmark
35+
arguments: "--test-time 180 -c 4 -t 1 --hide-histogram --key-prefix 'idx10:' --key-minimum 1 --key-maximum 100000 --command 'FT.SEARCH idx10 \"(( @field3: (1) @field6: [-inf 76.761 ] @field7: [76.761 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] )| ( @field3: (2) @field6: [-inf 137.894 ] @field7: [137.894 +inf ] @field4: [-inf 35323000 ] @field5: [35323000 +inf ] @field8: [-inf 1864.5 ] @field9:[1864.5 +inf ] ))\"' --command-ratio 90 --command 'HEXPIRE __key__ 1000 FIELDS 2 field6 field3' --command-ratio 10"

0 commit comments

Comments
 (0)