GEOWAVE-546 & GEOWAVE-696 Fixes / Updates#759
Merged
rwgdrummer merged 3 commits intolocationtech:0.9.1from May 7, 2016
Merged
Conversation
Suite - Change Kafka Driver to return list of Futures for topics, users can then wait on these until the topic times out - Change Kakfa Test Environment to Timeout the Consumer after it is done reading the IT data. After 5 seconds, it will timeout, and the rest of the IT will execute. Removed 60 second sleep.
| final KafkaIngestRunData ingestRunData, | ||
| final List<String> queue ) | ||
| throws Exception { | ||
| final ExecutorService executorService = Executors.newFixedThreadPool(queue.size()); |
Contributor
Author
There was a problem hiding this comment.
I just realized something here.. Is this creating a thread pool of X size for every single topic? Like if I had 3 topics it would create 3 threadpools of 3 max threads each? I think it would probably only actually create 3 threads but there might be overhead from the extra pools...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For GEOWAVE #546: If a kafka properties file isn't selected, then it won't apply command line options to the kafka config properties. Changed it so that it will do it whether the config file is specified or not.
For GEOWAVE #696: Fixed IndexCallbackList to implement Flushable, which will then allow the IndexWriter.flush() method to flush properly. Also, modified IT tearDown code to work in reverse order. For Kafka IT, it was tearing down Accumulo (with zookeeper) before tearing down kafka, and the kafka client was in an infinite loop trying to re-connect to zookeeper at the end of the IT.
Also, modified the BasicKafkaIT to not wait randomly 60 seconds, but instead to wait until all the data has been ingested, and the consumer times out before continuing the integration test. This will allow different systems with different performance specs to run the IT reliably. Added it back to GeoWaveITSuite.