Add ignored_fields to test config#82
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
|
| "ignored_fields": [ | ||
| "foobar", | ||
| "world", | ||
| "hello" | ||
| ] |
There was a problem hiding this comment.
I read through the use case for this PR: elastic/elastic-package#169. My understanding is that there will be some fields in the indexed documents that will have dynamic values - values whose value will be different each time the test is executed. However, these fields' values should still conform to a specific data type and shape.
So I would not recommend that we completely allow certain fields to be ignored. Instead, perhaps we can allow users to specify certain fields in the test config and specify a valid pattern for that field's value. So something like:
| "ignored_fields": [ | |
| "foobar", | |
| "world", | |
| "hello" | |
| ] | |
| "dynamic_fields": { | |
| "foobar": ".*", | |
| "a_numeric_field": "\\d+" | |
| } |
I'm not married to the name of this setting; just using dynamic_fields as a suggestion above.
This way we can allow for dynamic values for specific fields without completely ignoring them. WDYT?
There was a problem hiding this comment.
However, these fields' values should still conform to a specific data type and shape.
This is a good argument for dynamic fields.
Instead, perhaps we can allow users to specify certain fields in the test config and specify a valid pattern for that field's value.
We'll need to update the pipeline test runner to make the diff more smarter (consider regular expressions), but I think it's possible.
Issue: elastic/elastic-package#169
This PR adds
ignored_fieldsproperty definition to the test config.