Comments (7)
Version 1.1 support synchronous timeouts (i.e. block a thread to wait for an RPC to complete for a certain amount of time, and give up waiting if the RPC is taking too long). Asynchronous timeouts need to be implemented.
For synchronous timeouts, just call join(timeout)
on the Deferred
of the RPC.
from asynchbase.
Asynchronous timeouts need to be implemented.
Can you give me a brain dump on this? I wish to work on it.
from asynchbase.
Sure. For asynchronous timeouts, we need a timer thread that takes care of all the timeouts. This should be implemented with Netty's HashedWheelTimer
. Every single RPC that is sent out would schedule a new TimerTask
and save the Timeout
for this RPC, probably along with its Deferred
. Then for each response that arrives from the network, we'd take the Timeout
object and cancel()
it. If a timeout occurs, we'd callback the Deferred
with an exception, say a RpcTimeoutException
(that would inherit from RecoverableException
).
There are a couple subtleties involved, such as properly handling the cases where the RPC times out and then immediately after that you get the response back from the network. In this case the response should just be silently ignored as no processing / handling of it can occur, because we already gave a RpcTimeoutException
to the Deferred
.
The HBase RPC protocol doesn't support timeouts and doesn't support canceling RPCs, so once something has been sent to the wire, we can't take it back.
from asynchbase.
This is helpful. Let me see where I can get with this.
from asynchbase.
@tsuna , finally I sat down and created a raw patch of what I understood you intended this to be done as.
I broke down your words for my understanding as follows:
- "For asynchronous timeouts, we need a timer thread that takes care of all the timeouts"
- We don't need to create a new timer. Reuse existing 'timer' (already of type HashedWheelTimer) instance in HBaseClient.
- "Every single RPC that is sent out would schedule a new TimerTask and save the Timeout for this RPC, probably along with its Deferred"
- Create a new field in HBaseRpc class, rpc_timeout of type TimeOut.
- "Then for each response that arrives from the network, we'd take the Timeout object and cancel() it."
- In decode method of RegionClient we receive responses. Once received response, we will call rpc.cancelTimeout method (a new method to be written in HBaseRpc) . This method will call timeout.cancel()
- "If a timeout occurs, we'd callback the Deferred with an exception, say a RpcTimeoutException (that would inherit from RecoverableException)."
- We can create a timer task with delay set as value of the desired rpc timeout. I am thinking this could be per rpc timeout value. TimerTask executes once that delay is passed. In run method we will throw the exception RpcTimeoutException. In the world of deferred this means do a callback on deferred with argument as an instance of RpcTimeoutException which inherits RecoverableException.
Here is the rough patch (by no means it is close ;-) ) . Also for now only Get is supporting rpc timeout.
Patch : https://gist.github.com/2950832
from asynchbase.
Please send a pull request, it's going to make it easier to chat about the code than with a Gist.
from asynchbase.
Will do.
from asynchbase.
Related Issues (20)
- Asynchbase MultiAction deserializes response from batch putRequest incorrectly HOT 1
- PR build failed with Java environment error HOT 1
- AssertionError: "Must not be used." HOT 1
- IN version2.3.0 HOT 1
- An exception was thrown by TimerTask. java.lang.OutOfMemoryError: Java heap space HOT 1
- Should buffering gets be possible? HOT 1
- When a scanner lease expires, it will retry request same regionserver endless. RS too busy! HOT 16
- can't set async wal?
- Mutation Attributes support
- Appends with no returns and few errors can break deserialization
- Can't re-write data correctly HOT 1
- Can you open chat room
- Why ConnectionResetException happen? HOT 1
- SparkStreaming with mapPartitions use client find error ”Too many open files“ HOT 1
- Comparison method violates its general contract
- org.hbase.async.NonRecoverableException: Too many attempts: OpenScannerRequest HOT 1
- How do i use the PageFilter
- Broken pipe error with "hbase.rpc.protection: privacy" HOT 1
- Can this support the kerberos authentication of hbase 2.2 HOT 1
- Not able to create jar
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from asynchbase.