danielywoo / fast-object-pool Goto Github PK

A lightweight partitioned object pool, you can use it to pool expensive objects like jdbc connections, thrift clients etc.

Home Page: http://danielw.cn/fast-object-pool/

License: Apache License 2.0

Java 100.00%

fast-object-pool's People

Stargazers

Watchers

fast-object-pool's Issues

graceful shutdown

add shutdown method with timeout to the pool

Performance stats

Collect metrics like

borrow/return/create/validate/destroy latency.
TPS/fail/success stats
pool size, active count, idle count

Expose MBean
Expose micrometer, optional spring boot actuator metrics and health support

Stop supporting jdk1.7

Since Jdk1.8 is already very popular, and no new projects are using 1.7.

The current scavenger scheme is thread-per-pool. I have a use case where I could have 100+ pools (caching byte[] of various lengths for image processing), and I'd prefer not to have 100 scavenger threads running.

Would an option of providing an ObjectPool with a ScheduledExecutorService upon construction work? The pool then would schedule its own scavenging and cancel the scheduled job upon shutdown. I'd be willing to submit a PR with this change (or another scheme you propose) if you're open to it.

add returnToPool method to Poolable

so you don't have to remember which pool to return when you have multiple pools

Unit test wanted

We need more unit tests

chain setters in PoolConfig

travis needed

Add benchmark graph to readme.md

Call back on scavenge

It would be nice to have a callback to the object or factory during a scavenge (instead of just the hardcoded expired checks). Then, for example, if I were pooling Jedis connections I could check there to see if the connection is broken and recycle it instead of doing it each time the object is borrowed.

fast-object-pool 并不较 commons-pool2 有极大的性能提升

地址：https://github.com/wyzssw/poolcompare
测试类：com.justdebugit.pool.compare.PoolCompare
5个连接，10个线程的场景测试如下:

Stormpot has consumed :
总消耗1202 ms
borrow消耗:16 ms

fastobjectpool has consumed :
总消耗1417 ms
borrow消耗:983 ms

commons-pool has consumed :
总消耗1403 ms
borrow消耗:839 ms


threadlocalpool has consumed :
总消耗1134 ms
borrow消耗:1 ms

fastobjectpool 经常报错:

	at cn.danielw.fop.ObjectPool.returnObject(ObjectPool.java:84)
	at cn.danielw.fop.Poolable.returnObject(Poolable.java:34)
	at com.justdebugit.pool.compare.PoolCompare$3.release(PoolCompare.java:145)
	at com.justdebugit.pool.compare.PoolCompare$3.release(PoolCompare.java:1)
	at com.justdebugit.pool.compare.PoolCompare$6.run(PoolCompare.java:232)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
	at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:350)
	at cn.danielw.fop.ObjectPool.returnObject(ObjectPool.java:79)
	... 9 more

stopping JDK8 support

Considering

starts with 3.0.0 and only supports JDK 11+ with the latest performance enhancement.
continues to support 2.x.x with JDK8+ with limited feature requests.

Risk condition causing thread blocked

Poolable<T> freeObject = subPool.getObjectQueue().poll();
if (freeObject == null) {
    // increase objects and return one, it will return null if reach max size
    subPool.increaseObjects(1);
    try {
        if (blocking) {
            freeObject = subPool.getObjectQueue().take();
        } else {
            freeObject = subPool.getObjectQueue().poll(config.getMaxWaitMilliseconds(), TimeUnit.MILLISECONDS);
            if (freeObject == null) {
                throw new RuntimeException("Cannot get a free object from the pool");
            }
        }
    } catch (InterruptedException e) {
        throw new RuntimeException(e); // will never happen
    }
}

Code in borrowObject() may cause the thread blocked if the increased object was immediately taken by another thread.

Although I knew I can use borrowObject(false) to solve the issue. But I still regard this as a multi-thread bug. The newly created object is ought to be consumed by its own thread.

More intelligent logics of borrowing objects

Is it possible to introduce better logics of populating objects? Not just 3-times retrying.. I use if for thrift-clients, where every connection behind client could be closed by server-side, so I face the problem where 3 times is too small amount of tries...

Scavenger cannot be disabled

The JavaDoc comment on PoolConfig.setScavengeIntervalMilliseconds suggests that it is possible to completely disable the scavenger by setting this property to zero:

set it to zero if you don't want to automatically shrink your pool. This is useful for fixed-size pool, or pools don't increase too much.

and ObjectPool respects this if it has been set in the PoolConfig. However the config setter method itself does not permit a zero parameter value:

java.lang.IllegalArgumentException: Cannot set interval too short (0), must be at least 5 seconds
	at cn.danielw.fop.PoolConfig.setScavengeIntervalMilliseconds(PoolConfig.java:80)

Endless loop with partition shutdown

fast-object-pool/src/main/java/cn/danielw/fop/ObjectPoolPartition.java

Line 95 in 62baf75

public synchronized int shutdown() {

If for any reason an object is leaked, the shutdown method will not terminate. Would it be possible to have a force shutdown method, or shutdown with a timeout option?

<dependency>
      <groupId>org.jacoco</groupId>
      <artifactId>jacoco-maven-plugin</artifactId>
      <version>0.8.6</version>
      <type>maven-plugin</type>
</dependency>

I'm not sure why it's necessary as a "normal", non-plugin dependency.

Here's a snippet from my invocation of "mvn dependency:tree":

[INFO] +- cn.danielw:fast-object-pool:jar:2.2.0:compile
[INFO] |  \- org.jacoco:jacoco-maven-plugin:maven-plugin:0.8.6:compile
[INFO] |     +- org.apache.maven:maven-plugin-api:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-model:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-artifact:jar:3.0:compile
[INFO] |     |  \- org.sonatype.sisu:sisu-inject-plexus:jar:1.4.2:compile
[INFO] |     |     \- org.sonatype.sisu:sisu-inject-bean:jar:1.4.2:compile
[INFO] |     |        \- org.sonatype.sisu:sisu-guice:jar:noaop:2.1.7:compile
[INFO] |     +- org.apache.maven:maven-core:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-settings:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-settings-builder:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-repository-metadata:jar:3.0:compile
[INFO] |     |  +- org.apache.maven:maven-model-builder:jar:3.8.4:compile
[INFO] |     |  |  +- javax.inject:javax.inject:jar:1:compile
[INFO] |     |  |  +- org.apache.maven:maven-builder-support:jar:3.8.4:compile
[INFO] |     |  |  \- org.eclipse.sisu:org.eclipse.sisu.inject:jar:0.3.5:compile
[INFO] |     |  +- org.apache.maven:maven-aether-provider:jar:3.0:runtime
[INFO] |     |  +- org.sonatype.aether:aether-impl:jar:1.7:compile
[INFO] |     |  |  \- org.sonatype.aether:aether-spi:jar:1.7:compile
[INFO] |     |  +- org.sonatype.aether:aether-api:jar:1.7:compile
[INFO] |     |  +- org.sonatype.aether:aether-util:jar:1.7:compile
[INFO] |     |  +- org.codehaus.plexus:plexus-interpolation:jar:1.14:compile
[INFO] |     |  +- org.codehaus.plexus:plexus-classworlds:jar:2.2.3:compile
[INFO] |     |  +- org.codehaus.plexus:plexus-component-annotations:jar:1.5.5:compile
[INFO] |     |  \- org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3:compile
[INFO] |     |     \- org.sonatype.plexus:plexus-cipher:jar:1.4:compile
[INFO] |     +- org.codehaus.plexus:plexus-utils:jar:3.0.22:compile
[INFO] |     +- org.apache.maven.shared:file-management:jar:1.2.1:compile
[INFO] |     |  +- org.apache.maven.shared:maven-shared-io:jar:1.1:compile
[INFO] |     |  |  +- org.apache.maven:maven-artifact-manager:jar:2.0.2:compile
[INFO] |     |  |  \- org.apache.maven.wagon:wagon-provider-api:jar:1.0-alpha-6:compile
[INFO] |     |  \- org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9:compile
[INFO] |     |     +- junit:junit:jar:4.13.2:compile
[INFO] |     |     |  \- org.hamcrest:hamcrest-core:jar:2.2:compile
[INFO] |     |     \- classworlds:classworlds:jar:1.1-alpha-2:compile
[INFO] |     +- org.apache.maven.reporting:maven-reporting-api:jar:3.0:compile
[INFO] |     |  \- org.apache.maven.doxia:doxia-sink-api:jar:1.0:compile
[INFO] |     +- org.apache.maven.reporting:maven-reporting-impl:jar:2.1:compile
[INFO] |     |  +- org.apache.maven:maven-project:jar:2.0.10:compile
[INFO] |     |  |  +- org.apache.maven:maven-profile:jar:2.0.10:compile
[INFO] |     |  |  \- org.apache.maven:maven-plugin-registry:jar:2.0.10:compile
[INFO] |     |  +- org.apache.maven.doxia:doxia-core:jar:1.1.2:compile
[INFO] |     |  |  +- org.apache.maven.doxia:doxia-logging-api:jar:1.1.2:compile
[INFO] |     |  |  +- xerces:xercesImpl:jar:2.8.1:compile
[INFO] |     |  |  +- commons-lang:commons-lang:jar:2.4:compile
[INFO] |     |  |  \- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] |     |  +- org.apache.maven.doxia:doxia-site-renderer:jar:1.1.2:compile
[INFO] |     |  |  +- org.apache.maven.doxia:doxia-decoration-model:jar:1.1.2:compile
[INFO] |     |  |  +- org.apache.maven.doxia:doxia-module-xhtml:jar:1.1.2:compile
[INFO] |     |  |  +- org.apache.maven.doxia:doxia-module-fml:jar:1.1.2:compile
[INFO] |     |  |  +- org.codehaus.plexus:plexus-i18n:jar:1.0-beta-7:compile
[INFO] |     |  |  +- org.codehaus.plexus:plexus-velocity:jar:1.1.7:compile
[INFO] |     |  |  +- org.apache.velocity:velocity:jar:1.5:compile
[INFO] |     |  |  \- commons-collections:commons-collections:jar:3.2:compile
[INFO] |     |  \- commons-validator:commons-validator:jar:1.2.0:compile
[INFO] |     |     +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] |     |     +- commons-digester:commons-digester:jar:1.6:compile
[INFO] |     |     +- commons-logging:commons-logging:jar:1.0.4:compile
[INFO] |     |     +- oro:oro:jar:2.0.8:compile
[INFO] |     |     \- xml-apis:xml-apis:jar:1.0.b2:compile
[INFO] |     +- org.jacoco:org.jacoco.agent:jar:runtime:0.8.6:compile
[INFO] |     +- org.jacoco:org.jacoco.core:jar:0.8.6:compile
[INFO] |     |  +- org.ow2.asm:asm:jar:8.0.1:compile
[INFO] |     |  +- org.ow2.asm:asm-commons:jar:8.0.1:compile
[INFO] |     |  |  \- org.ow2.asm:asm-analysis:jar:8.0.1:compile
[INFO] |     |  \- org.ow2.asm:asm-tree:jar:8.0.1:compile
[INFO] |     \- org.jacoco:org.jacoco.report:jar:0.8.6:compile

This is a work-around users can add to their own POMs:

            <dependency>
                <groupId>cn.danielw</groupId>
                <artifactId>fast-object-pool</artifactId>
                <version>2.2.0</version>
                <exclusions>
                    <exclusion>
                        <groupId>org.jacoco</groupId>
                        <artifactId>jacoco-maven-plugin</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>

release for both JDK8, and 11+

As the new JDKs have new optimizations, and the JDK8 is EOL, consider different release strategies for new JDKs while keep JDK8 users happy.

danielywoo / fast-object-pool Goto Github PK

fast-object-pool's People

Stargazers

Watchers

Forkers

fast-object-pool's Issues

Recommend Projects

Recommend Topics

Recommend Org