Repro steps:
Create index with following setting:
POST / myindex
{
"settings" : {
"analysis" : {
"filter" : {
"customsplit" : {
"type" : "pattern_capture",
"preserve_original" : 1,
"patterns" : [
"([^_.]+)"
]
}
},
"analyzer" : {
**"standard"** : {
"tokenizer" : "standard",
"filter" : [
"lowercase",
"customsplit"
]
}
}
}
},
"mappings" : {
"docs" : {
"properties" : {
"Url" : {
"type" : "string"
}
}
}
}
}
Please be noticed that:
- The analyzer is not defined for Url field, so it should use default analyzer which is "standard" analyzer.
- In myindex setting, "standard" analyzer is re-written by adding "customsplit" filter in addition to "lowercase" filter
- The customsplit filter is a pattern_capture filter to split word by "." and "_", which allows the re-written standard analyzer split DNS to several tokens as following output:
GET /myindex/_analyze?analyzer=standard&text=www.xyz.com
{
"tokens": [
{
"token": "www.xyz.com",
"start_offset": 0,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "www",
"start_offset": 0,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "xyz",
"start_offset": 0,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "com",
"start_offset": 0,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
}
]
}
Insert one doc into myindex:
POST /myindex/docs/1
{
"Url": "www.xyz.com"
}
Search "xyz" on "Url" field:
POST /myindex/_search
{
"query": {
"match": {
"Url": "xyz"
}
}
}
Actual Result: None document returned,
Expected Result: doc 1 should be returned.
Update the index setting by explicitly specifying "Url" field analyzer to "standard:
{
mappings": {
"docs": {
"properties": {
"Url": {
"type": "string",
**"analyzer": "standard"**
}
}
}
}
Search "xyz" on Url field with exact same query DSL in step 3
Actual result; doc 1 returned as expected.
I guess for default analyzer, ES just pick up the built-in analyzer, it is built-in "standard" in my case, regardless whether the analyzer is re-written in index setting.
It is somehow confusing. I think it should either reject "standard" as an analyzer name in customized analyzer setting or pick the right one when indexing the document.
You can also find the detailed issue discussion in elasticsearch forum: https://discuss.elastic.co/t/what-is-default-index-analyzer/69402/4
Repro steps:
Create index with following setting:
Please be noticed that:
Insert one doc into myindex:
Search "xyz" on "Url" field:
Actual Result: None document returned,
Expected Result: doc 1 should be returned.
Update the index setting by explicitly specifying "Url" field analyzer to "standard:
Search "xyz" on Url field with exact same query DSL in step 3
Actual result; doc 1 returned as expected.
I guess for default analyzer, ES just pick up the built-in analyzer, it is built-in "standard" in my case, regardless whether the analyzer is re-written in index setting.
It is somehow confusing. I think it should either reject "standard" as an analyzer name in customized analyzer setting or pick the right one when indexing the document.
You can also find the detailed issue discussion in elasticsearch forum: https://discuss.elastic.co/t/what-is-default-index-analyzer/69402/4