-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Is your feature request related to a problem? Please describe.
ES users create indexes by hour, day, month etc such as log_2020_6_20、 log_2020_6_21、log_2020_6_22 .., At this time, they can easily search all those index by using aliases and wildcards to match those indexes
Currently Doris On ES does not support create external table with aliases or wildcard index such as log_*, if you create such table would cause unpredictable results, this may annoy many Doris On ES users which is what we don't want to see
Describe the solution you'd like
After this PR #3454 (comment) was merged into Doris (This provides great convenience for supporting alias and wildcard indexes, although I am not satisfied with the implementation inside, I will rename and refactor later)
- obtain the
mappingfrom{index}/_mappingAPI - fetch all resolved index shard partitions from
{index}/_search_shards - split all shard partitions with corresponding index, Doris BE execute all
_searchconcurrently
_mapping and _search_shards support wildcard and aliases indexName
Additional context
Please Note: we just validate the first index from _mapping API, so DOE's user should guarantee all involved index has same schema or fields at least now( this maybe changed in the future)