-
-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Fix inconsistent counter resets due to Config::purge()
#8635
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The new files created in this change contain |
8fa93c7 to
4d50d1a
Compare
4d50d1a to
ad79c33
Compare
osquery/utils/affixes.cpp
Outdated
| @@ -0,0 +1,31 @@ | |||
| /** | |||
| * Copyright (c) 2025-present, Akamai Technologies Inc. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skurpad7 thank you for asking about this! We will need the standard osquery copyright header which says
/**
* Copyright (c) 2014-present, The osquery authors
*
* This source code is licensed as defined by the LICENSE file found in the
* root directory of this source tree.
*
* SPDX-License-Identifier: (Apache-2.0 OR GPL-2.0-only)
*/
You will also need to sign the CLA.
Note that the "check code style" action (https://github.com/osquery/osquery/actions/runs/15977093557/job/45232527575?pr=8635) is failing because of this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, thank you! I have updated the header accordingly.
|
@skurpad7 we need you to sign the CLA in order to merge any code to the project. Thank you! |
|
Hi @zwass! the CLA has been signed. |
|
Reopened PR to kick off CI. |
zwass
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@skurpad7 thank you so much for the thoroughly documented issue, PR, and code. I'm very pleased to review something with such attention to detail.
Thank you for your patience awaiting this review.
I intend to merge this once CI passes.
|
Awesome! thank you so much for the review and the guidance through the PR, @zwass! |
Fixes #8611
As described in the above issue, Osquery stores the state of the scheduled queries in the
kQueriesdomain in RocksDB. TheConfig::purge()function is invoked on every config refresh. It is responsible for purging the stored state of queries that are no longer in the schedule and haven't been executed in the last 7 days. Whenever a query is scheduled, the timestamp for it is updated astimestamp.<name>in thekPersistentSettingsdomain. TheConfig::purge()function however, sets the timestamps for all metadata keys in thekQueriesdomain such as<name>counter,<name>epoch,query.<name>as well if not already present. The Osquery scheduler never updates the timestamp for these metadata keys. Every 7 days,Config::purge()checks the timestamps of these keys, and because they always have week old timestamps, they are purged from the store. This causes the counter metadata key of queries to reset but only differential results are reported leading to the issue above.With this change,
1. Metadata keys are ignored from the purging logic.
2. Users will not be allowed to add queries with
query.,cache.andconfig_views.as a prefix orepochandcounteras a suffix.Config module changes
The
queryExistsfunction returnsfalsefor queries even if they are present in the schedule. Consider a queryquery_namein a packpack_namewith/as the pack delimiter. WhenqueryExists("pack/pack_name/query_name")is called, it iterates over the schedule and compares if"query_name" == "pack/pack_name/query_name". This always returns false even if the query is present in the schedule. This function is updated to use fully qualified query names to fix this behavior.The timestamp has the final say in determining if a key is purged from the store. The
Config::purge()function ages and purges reserved keys such as<name>counter,<name>epochandquery.<name>as described above. This leads to the counter (and other query metadata) being purged inconsistently. With this change, such keys are not considered for purge. The unit testtest_config_results_purgeis updated to validate the same.Currently, there are no restrictions on the names that can be set for a query. Query names can contain metadata prefixes such as
query.andcache.or suffixes such asepochandcounter. This leads to the following issues:pack1, if there are two queries with namesquery1andquery1counter, osquery crashes with the following error:E0624 15:50:45.187048 119689 shutdown.cpp:80] Error adding new results to database for query pack/pack1/query1counter: JSON object was not an array.This change prevents queries from having such reserved prefixes and suffixes in their names to address the above issues and adds a new unit test
test_queryname_validationfor the same.Database module changes
kReservedPrefixesandkReservedSuffixesare defined to store the reserved prefixes and suffixes that have a special meaning in the RocksDB store.Util Module Changes
bool hasAnyPrefix(const std::string& s, const std::vector<std::string>& prefixes),bool hasAnySuffix(const std::string& s, const std::vector<std::string>& suffixes)and their corresponding unit testtest_affixesare added. These functions are used in the config module to check if a string has any of the reserved database prefixes or suffixes.