Error uploading data: KeyValue size too large
How to set hbase. client. keyvalue. maxsize parameter of HBase?
The following code settings don't work either.
HBaseOptions thbaseOptions=new HBaseOptions();
HBaseOperations operations = new HBaseOperations(this.zookeeper,this.tableName,thbaseOptions);
System.out.println("old hbase.client.keyvalue.maxsize="+operations.getConnection().getConfiguration().get("hbase.client.keyvalue.maxsize"));
//display value(old hbase.client.keyvalue.maxsize=10485760) //10M
operations.getConnection().getConfiguration().set("hbase.client.keyvalue.maxsize","1073741824"); //1024M
HBaseDataStore dataStore = new HBaseDataStore(operations,thbaseOptions);
Configuration config=operations.getConnection().getConfiguration();
//config.set("hbase.client.keyvalue.maxsize","1073741824"); //1024M
//dataStore.prepareRecordWriter(config);
System.out.println("new hbase.client.keyvalue.maxsize="+config.get("hbase.client.keyvalue.maxsize"));
//display value(new hbase.client.keyvalue.maxsize=1073741824)
IndexWriter writer = dataStore.createWriter(featureDataAdapter, primaryIndex);
The error message is as follows:
Exception in thread "main" java.lang.IllegalArgumentException: KeyValue size too large at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1545) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.validatePut(BufferedMutatorImpl.java:175) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:146) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.writeMutations(HBaseWriter.java:94) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:88) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:81) at mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:99) at mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:72) at mil.nga.giat.geowave.core.store.index.writer.IndexCompositeWriter.write(IndexCompositeWriter.java:49) at com.cwgis.importFeatureHBase.importData(importFeatureHBase.java:120) at com.cwgis.App.import_db(App.java:115) at com.cwgis.App.main(App.java:51)
Error uploading data: KeyValue size too large
How to set hbase. client. keyvalue. maxsize parameter of HBase?
The following code settings don't work either.
HBaseOptions thbaseOptions=new HBaseOptions();
HBaseOperations operations = new HBaseOperations(this.zookeeper,this.tableName,thbaseOptions);
System.out.println("old hbase.client.keyvalue.maxsize="+operations.getConnection().getConfiguration().get("hbase.client.keyvalue.maxsize"));
//display value(old hbase.client.keyvalue.maxsize=10485760) //10M
operations.getConnection().getConfiguration().set("hbase.client.keyvalue.maxsize","1073741824"); //1024M
HBaseDataStore dataStore = new HBaseDataStore(operations,thbaseOptions);
Configuration config=operations.getConnection().getConfiguration();
//config.set("hbase.client.keyvalue.maxsize","1073741824"); //1024M
//dataStore.prepareRecordWriter(config);
System.out.println("new hbase.client.keyvalue.maxsize="+config.get("hbase.client.keyvalue.maxsize"));
//display value(new hbase.client.keyvalue.maxsize=1073741824)
IndexWriter writer = dataStore.createWriter(featureDataAdapter, primaryIndex);
The error message is as follows:
Exception in thread "main" java.lang.IllegalArgumentException: KeyValue size too large at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:1545) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.validatePut(BufferedMutatorImpl.java:175) at org.apache.hadoop.hbase.client.BufferedMutatorImpl.mutate(BufferedMutatorImpl.java:146) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.writeMutations(HBaseWriter.java:94) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:88) at mil.nga.giat.geowave.datastore.hbase.operations.HBaseWriter.write(HBaseWriter.java:81) at mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:99) at mil.nga.giat.geowave.core.store.base.BaseIndexWriter.write(BaseIndexWriter.java:72) at mil.nga.giat.geowave.core.store.index.writer.IndexCompositeWriter.write(IndexCompositeWriter.java:49) at com.cwgis.importFeatureHBase.importData(importFeatureHBase.java:120) at com.cwgis.App.import_db(App.java:115) at com.cwgis.App.main(App.java:51)