Skip to content

GEOWAVE-441. Improve performance of WFS-T with small transaction set…#443

Merged
rfecher merged 2 commits intomasterfrom
GEOWAVE-441
Jul 6, 2015
Merged

GEOWAVE-441. Improve performance of WFS-T with small transaction set…#443
rfecher merged 2 commits intomasterfrom
GEOWAVE-441

Conversation

@rwgdrummer
Copy link
Copy Markdown
Contributor

…s (e.g. < 10000) using buffer and bulk writes

@rwgdrummer
Copy link
Copy Markdown
Contributor Author

This approach retains current functionality while gaining performance through batch insertions to the DataStore. This approach does not require any additional work/configuration from the users/administrators.

(e.g. < 10000) using buffer and bulk writes.  Allow the buffer size to be
configured.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commenting format is wonderful :) :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad you like it. So much for auto-formatting. I fixed this (twice...annoyingly enough).

@chrisbennight
Copy link
Copy Markdown
Contributor

Gotcha - I was confused the first time we talked I think; I thought you were replacing the visibility based transaction isolation with just a local in memory map. (which isn't at all what's happening)

Is it fair to say that this is jut buffering the contents of transactions in an in memory structure and only flushing them to the datastore when flush is called - or when they have exeeded the max buffer size?

So really it would be of greatest benefit to largest transaction sets, not small (i.e. if the transaction set was ~1 it would be similar to previous behavior?)

@rwgdrummer
Copy link
Copy Markdown
Contributor Author

Not identical to previous behavior. For small transactions, the visibility transformer is never used. The transformer is costly and not required in the bulk-write scenario. Thus, the benefit to any transaction, where the transactions fit in the buffer, is efficient bulk-writes without the overhead of transforming the visibility. This is the typical use case. Thus, users that are writing small batches of transactions gain immediate benefit without changing their code or configuration.

The benefit to large transactions that exceed the buffer size is the efficient bulk-writes.
All the while, transaction isolation is still maintained in those cases.

rfecher added a commit that referenced this pull request Jul 6, 2015
GEOWAVE-441.  Improve performance of WFS-T with small transaction set…
@rfecher rfecher merged commit ff3f0c9 into master Jul 6, 2015
@rfecher rfecher deleted the GEOWAVE-441 branch July 6, 2015 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants