GEOWAVE-441. Improve performance of WFS-T with small transaction set…#443
GEOWAVE-441. Improve performance of WFS-T with small transaction set…#443
Conversation
|
This approach retains current functionality while gaining performance through batch insertions to the DataStore. This approach does not require any additional work/configuration from the users/administrators. |
(e.g. < 10000) using buffer and bulk writes. Allow the buffer size to be configured.
There was a problem hiding this comment.
Commenting format is wonderful :) :)
There was a problem hiding this comment.
Glad you like it. So much for auto-formatting. I fixed this (twice...annoyingly enough).
|
Gotcha - I was confused the first time we talked I think; I thought you were replacing the visibility based transaction isolation with just a local in memory map. (which isn't at all what's happening) Is it fair to say that this is jut buffering the contents of transactions in an in memory structure and only flushing them to the datastore when flush is called - or when they have exeeded the max buffer size? So really it would be of greatest benefit to largest transaction sets, not small (i.e. if the transaction set was ~1 it would be similar to previous behavior?) |
…ting supporting documentation.
|
Not identical to previous behavior. For small transactions, the visibility transformer is never used. The transformer is costly and not required in the bulk-write scenario. Thus, the benefit to any transaction, where the transactions fit in the buffer, is efficient bulk-writes without the overhead of transforming the visibility. This is the typical use case. Thus, users that are writing small batches of transactions gain immediate benefit without changing their code or configuration. The benefit to large transactions that exceed the buffer size is the efficient bulk-writes. |
GEOWAVE-441. Improve performance of WFS-T with small transaction set…
…s (e.g. < 10000) using buffer and bulk writes