larsgeorge / hbase-book Goto Github PK
View Code? Open in Web Editor NEWContains the code used in the HBase: The Definitive Guide book.
Home Page: http://hbasebook.com
License: Apache License 2.0
Contains the code used in the HBase: The Definitive Guide book.
Home Page: http://hbasebook.com
License: Apache License 2.0
Hi Lars,
The compilation of all the source files in the directory is successful after the issual of
the mvn package command.
INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 seconds
[INFO] Finished at: Fri Jul 06 17:21:33 CDT 2012
[INFO] Final Memory: 65M/380M
[INFO] ------------------------------------------------------------------------
However when I try to run any of the examples, errors pop up.
varun@varun:~/larsgeorge/hbase-book/ch04$ bin/run.sh client.PutExample
Exception in thread "main" java.lang.NoClassDefFoundError: client/PutExample
Caused by: java.lang.ClassNotFoundException: client.PutExample
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
What is the issue?Could you please provide your inputs on this ?
K[] samples = sampler.getSample(inf, job);
use jdk 1.8
error: incompatible types.
Required: K[]
Found: java.lang.Object[]
auto cast:
K[] samples = (K[]) sampler.getSample(inf, job);
I have written a custom Filter extending FilterBase and converted into JAR. The filter looks like this :
public class MyFilter extends FilterBase {
boolean filterRow= true;
String srh;
public MyFilter(String str) {
this.srh= str;
}
@Override
public ReturnCode filterKeyValue(Cell c) throws IOException {
String str= Bytes.toString(c.getValue());
if(str.contains(str)) {
filterRow= false;
return ReturnCode.INCLUDE;
}
filterRow= true;
return ReturnCode.SKIP;
}
@Override
public boolean filterRow() {
return filterRow;
}
}
After making the JAR file (i.e. MyFilter.jar) I put it in /use/local/HBase/lib/filters directory. Then I set
export HBASE_CLASSPATH="/usr/local/Hbase/lib/filters/MyFilter.jar"
in hbase-env.sh filter and restarted the hbase server. Then I used the custom filter from java program :
public static void main(String argv[]) throws IOException {
Configuration conf= HBaseConfiguration.create();
Connection con= ConnectionFactory.createConnection(conf);
Table table= con.getTable(TableName.valueOf("stud"));
Filter fl= new MyFilter("uc");
Scan sc= new Scan();
sc.setFilter(fl);
ResultScanner rs= table.getScanner(sc);
for(Result r : rs)
System.out.println(Bytes.toString(r.getValue(Bytes.toBytes("perData"), Bytes.toBytes("name"))));
}
But getting following exception
Exception in thread "main" org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1478)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:993)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2396)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1474)
... 8 more
Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException: parseFrom called on base Filter, but should be called on derived type
at org.apache.hadoop.hbase.filter.Filter.parseFrom(Filter.java:270)
... 13 moreat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:329)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:408)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:204)
at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:65)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:210)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:364)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:338)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:65)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1478)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:993)
at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2396)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33648)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toFilter(ProtobufUtil.java:1474)
... 8 more
Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException: parseFrom called on base Filter, but should be called on derived type
at org.apache.hadoop.hbase.filter.Filter.parseFrom(Filter.java:270)
... 13 moreat org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1267)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:34094)
at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:400)
... 10 more
Even after doing mvn package i dont see run.sh coming in bin folder neither it was present in the same when i cloned.
How to solve the problem of importing data failure when importing mysql data into hbase (version 2.1.2 or 2.1.3) using sqoop (version 1.4.7)
the code in line 93 from ImportFromFile
context.write(new ImmutableBytesWritable(rowkey), put);
has syntax error:
The method write(ImmutableBytesWritable, Writable) in the type TaskInputOutputContext<LongWritable,Text,ImmutableBytesWritable,Writable> is not applicable for the arguments (ImmutableBytesWritable, Put).
by the way, my hadoop-core version is 1.2.1 because I can't get 0.20-append-r1057313 version by maven:
I don't know what's going wrong,can you help me?
INFO main - Client environment:java.library.path=.:/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java
INFO main - Client environment:java.io.tmpdir=/var/folders/g5/08jclb4s7tg4swz4jx31h2sc0000gn/T/
INFO main - Client environment:java.compiler=
INFO main - Client environment:os.name=Mac OS X
INFO main - Client environment:os.arch=x86_64
INFO main - Client environment:os.version=10.9.2
INFO main - Client environment:user.name=yuchao
INFO main - Client environment:user.home=/Users/yuchao
INFO main - Client environment:user.dir=/Users/yuchao/Downloads/hbase-book/hush
INFO main - Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
INFO main-SendThread() - Opening socket connection to server localhost/fe80:0:0:0:0:0:0:1%1:2181
INFO main-SendThread(localhost:2181) - Socket connection established to localhost/fe80:0:0:0:0:0:0:1%1:2181, initiating session
INFO main-SendThread(localhost:2181) - Session establishment complete on server localhost/fe80:0:0:0:0:0:0:1%1:2181, sessionid = 0x144d2f6553a0008, negotiated timeout = 40000
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1937)
at org.apache.hadoop.hbase.ServerName.parseHostname(ServerName.java:81)
at org.apache.hadoop.hbase.ServerName.(ServerName.java:63)
at org.apache.hadoop.hbase.MasterAddressTracker.getMasterAddress(MasterAddressTracker.java:62)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:568)
at org.apache.hadoop.hbase.client.HBaseAdmin.(HBaseAdmin.java:94)
at com.hbasebook.hush.schema.SchemaManager.process(SchemaManager.java:126)
at com.hbasebook.hush.HushMain.main(HushMain.java:57)
Hi.. I am getting this error when I am trying to parse json...
with follwoing flags
hadoop jar hbasech07.jar ParseJson -i testtable -o testtable
throws this error:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
... 7 more
Caused by: java.lang.NoClassDefFoundError: org/json/simple/parser/JSONParser
at mapreduce.ParseJson$ParseMapper.(ParseJson.java:49)
... 12 more
Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.JSONParser
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
... 13 more
Hi
How to connect hbase remotely? what config i need to change in hbase server? thanks
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "192.168.0.174");
conf.set("hbase.zookeeper.property.clientPort", "2181");
HBaseAdmin.available(conf);
Above doesn't work
thanks
Peter
As version 2.0.0,I can not use hbase hbck (version 1) to fix my data inconsistencies, because -fix/ -repair options are NOT supported in version 2.0+ . I can not use hbase hbck2 (version2) to fix ,because it need hbase in bigger than version 2.0.3. It is crazy for me, anyone can solve it.
Excuse me, in the HBase1.3 version, how is the row lock explicitly used; the original HTable's lockRow () method and the unlockRow () method seem to have been lost
Could you please add an explicit LICENSE file to the repo so that it's clear under what terms the content is provided, and under what terms user contributions are licensed?
Generally speaking, the absence of a license means that the default copyright laws apply. This means that you retain all rights to your source code and that nobody else may reproduce, distribute, or create derivative works from your work. This might not be what you intend.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.