Skip to content

GEOWAVE-157#367

Merged
rfecher merged 1 commit intomasterfrom
GEOWAVE-157-postRefactor
May 5, 2015
Merged

GEOWAVE-157#367
rfecher merged 1 commit intomasterfrom
GEOWAVE-157-postRefactor

Conversation

@dcy2003
Copy link
Copy Markdown
Contributor

@dcy2003 dcy2003 commented May 5, 2015

Implementation for #157 ...

The crux of the solution is within AccumuloKeyValuePairGenerator which produces Accumulo [Key,Value] pairs (Geowave-formatted) given an adapter and an index, similar to the way we currently produce Mutations.

To fully demonstrate the "flow" of this solution, I created an example MapReduce job that:

  1. reads in some sample Geonames data (downloaded a bunch of points in Barbados)
  2. constructs SimpleFeatures for the sample data
  3. leverages AccumuloKeyValuePairGenerator to generate [Key,Value] pairs
  4. writes the data to file using AccumuloFileOutputFormat (after MapReduce sorts the [Key,Value] pairs)

Feedback welcome!

@rfecher
Copy link
Copy Markdown
Contributor

rfecher commented May 5, 2015

I looked through the files, and it looks like a good example. If we choose to follow-on with this to add more re-usable components, one thought would be to try to minimize the constants defined near the top of SimpleFeatureToAccumuloKeyValueMapper to provide a re-usable mapper that can be given the index and adapter through the job context, and the key generator could be more hidden so that the implementor could call something like writeNativeObject and the key generator would then take the native object and write the collection of key value pairs to the context...anyways, just a thought, not sure if its worth putting more into this yet.

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.17%) to 63.18% when pulling 6ea5160 on GEOWAVE-157-postRefactor into fc9df6c on master.

rfecher added a commit that referenced this pull request May 5, 2015
@rfecher rfecher merged commit 9bedebc into master May 5, 2015
@rfecher rfecher deleted the GEOWAVE-157-postRefactor branch May 5, 2015 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants