Event PubSub topics + linear filtering.#18266
Conversation
|
Nice! Thanks @calavera |
|
Didn't know that we have crappy |
05ca2f0 to
3181618
Compare
|
@LK4D4 we can probably change it. That That way, the events interface in the daemon could be something like: type eventsBackend interface {
Subscribe(since, int64, eventFilter filterArgs) ([]interface{}, chan interface{})
Unsubscribe(listener chan interface{})
} |
692645b to
9c2d27c
Compare
|
@LK4D4 that |
Improves the current filtering implementation complixity. Currently, the best case is O(N) and worst case O(N^2) for key-value filtering. In the new implementation, the best case is O(1) and worst case O(N), again for key-value filtering. Signed-off-by: David Calavera <david.calavera@gmail.com>
9c2d27c to
40c5e05
Compare
pkg/pubsub/publisher.go
Outdated
There was a problem hiding this comment.
You can eliminate else now :)
A TopicFunc is an interface to let the pubisher decide whether it needs to send a message to a subscriber or not. It returns true if the publisher must send the message and false otherwise. Users of the pubsub package can create a subscriber with a topic function by calling `pubsub.SubscribeTopic`. Message delivery has also been modified to use concurrent channels per subscriber. That way, topic verification and message delivery is not o(N+M) anymore, based on the number of subscribers and topic verification complexity. Using pubsub topics, the API stops controlling the message delivery, delegating that function to a topic generated with the filtering provided by the user. The publisher sends every message to the subscriber if there is no filter, but the api doesn't have to select messages to return anymore. Signed-off-by: David Calavera <david.calavera@gmail.com>
40c5e05 to
434d2e8
Compare
|
LGTM |
2 similar comments
|
LGTM |
|
LGTM |
Event PubSub topics + linear filtering.
|
Gah - please be careful - your changing/breaking docker remote API when changing raw API types (which affects docker compose and other tools), and there's been no update to the docs (API or release notes) for this change: Old remote API format: |
|
@twhiteman filters are backwards compatible and the api knows how to deserialize both formats. See https://github.com/docker/engine-api/blob/master/types/filters/parse.go#L93-L115. Please, feel free to open an issue if you bumped into problems. |
Although these two changes could go separatedly, I rather put them together because they are very related. Our filtering system is very inneficient at matching results. This can slow down the events API with complicated filters. Moreover, that API performs operations in linear complexity at best, and exponencial at worst, when we can do constant time at best and linear at worst.
I separated both commits and wrote explanation of the changes in each of them, but I'm copying both messages here for easy reading: