Giter VIP home page Giter VIP logo

blazingcache's People

Contributors

aluccaroni avatar caliuf avatar diegosalvi avatar dmercuriali avatar eolivelli avatar fomkin avatar hamadodene avatar mcsuperroot avatar mino181295 avatar niccomlt avatar nicoloboschi avatar pv3nturi avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

dianacle eolivelli mcroot edoardoantonini722 nicoloboschi fomkin caliuf mino181295 pv3nturi apurbad sp2020jarvan dmercuriali aluccaroni ghostboyboy misselvexu hamadodene tiny-rain niccomlt

blazingcache's Issues

Local locks lead to frequent deadlocks

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-25

Reporter: eolivelli @eolivelli

The fix for BLAZ-24, which introduced local locks in order to serialize access to entries locally, introduced a new race condition which leads to deadlock (resolved automaticallly by timeouts)

The case is the following:

  • client1 has the entry in memory
  • client1 starts and invalidation and locks locally the entry
  • client2 starts a put on the same entry
  • the put arrives on the server before the invalidation and acquires the lock on the entry
  • the put tries to send the put message to client1, which in turn already locked locally the key
  • the put on client1 wait for the completion of the invalidation, which is waiting on the server for the completion of the put

deadlock !!

Comments from JIRA

Enrico Olivelli 2016-03-25T10:54:37.000+0100

merged #53

On Jenkins Kerberos tests are failing

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-63

Reporter: eolivelli @eolivelli

{code:java}
java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at blazingcache.security.sasl.SaslNettyClient.evaluateChallenge(SaslNettyClient.java:114)
at blazingcache.security.sasl.ClientAuthenticationUtils.performAuthentication(ClientAuthenticationUtils.java:44)
at blazingcache.network.netty.GenericNettyBrokerLocator.connect(GenericNettyBrokerLocator.java:88)
at blazingcache.client.CacheClient.connect(CacheClient.java:322)
at blazingcache.client.CacheClient.access$300(CacheClient.java:59)
at blazingcache.client.CacheClient$ConnectionManager.run(CacheClient.java:360)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at blazingcache.security.sasl.SaslNettyClient$2.run(SaslNettyClient.java:116)
at blazingcache.security.sasl.SaslNettyClient$2.run(SaslNettyClient.java:114)
... 9 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)
at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:770)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
... 11 more
Caused by: KrbException: Server not found in Kerberos database (7) - Server not found in Kerberos database
at sun.security.krb5.KrbTgsRep.(KrbTgsRep.java:73)
at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:259)
at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:270)
at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:302)
at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:120){code}

Handle ZooKeeper session expiration

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-29

Reporter: eolivelli @eolivelli

In case of session expiration the ZK becomes unusable and the CacheServer cannot request leadership anymore.
So it can happen that in a cluster there is no cache leader at all

Comments from JIRA

Enrico Olivelli 2016-04-12T15:09:09.000+0200

IMHO the option zk.recoversession=true in server.properties is not useful

Enrico Olivelli 2016-04-12T15:10:49.000+0200

same on setupCluster

Load JCache Configuration from file (classpath)

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-50

Reporter: eolivelli @eolivelli

By default we are goping to load configuration from "blazingcache.properties"
use "configfile" property (from querystring or cachemanager properties) to override the location.

set configfile= "empty string" to disable the feature

the file will be loaded from the Classloader requested by the boot and from the Context Classloader (to enable configuration inside WARs)

Do not log full Exception in case of eviction when connection is not active

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-38

Reporter: alessandro.luccaroni @aluccaroni

It's better to write a more compact log, like the one that we write on invalidation:
{code}16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient invalidate
SEVERE: invalidate idc-3896#batchstatus22249, not connected{code}

Maybe something like:
{code}16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient ensureMaxMemoryLimit
SEVERE: evict idc-3021#n2747 size 54,649 bytes lastAccessDate 2,773,457,343,755,929, not connected{code}

Here a log from a production system:
{code}
16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient ensureMaxMemoryLimit
SEVERE: evict idc-3021#n2747 size 54,649 bytes lastAccessDate 2,773,457,343,755,929

16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient$3 replyReceived
SEVERE: error while unregistering entry idc-3021#n2747
java.lang.Exception: NettyChannel{name=10.200.86.81:1025, id=6, socket=null pending 0 msgs} connection is not active
at blazingcache.network.netty.NettyChannel.lambda$sendMessageWithAsyncReply$4(NettyChannel.java:188)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient ensureMaxMemoryLimit
SEVERE: evict idc-3021#n2748 size 57,364 bytes lastAccessDate 2,773,457,344,180,777

16-05-19-08-02-08 SEVERE May 19, 2016 8:02:08 AM blazingcache.client.CacheClient$3 replyReceived
SEVERE: error while unregistering entry idc-3021#n2748
java.lang.Exception: NettyChannel{name=10.200.86.81:1025, id=6, socket=null pending 0 msgs} connection is not active
at blazingcache.network.netty.NettyChannel.lambda$sendMessageWithAsyncReply$4(NettyChannel.java:188)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745){code}

JCache: abilitity to launch an Embedded CacheServer even in replicated mode (blazingcache.mode=server)

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-57

Reporter: eolivelli @eolivelli

The idea si to have an option to launch a CacheServer inside of every client in case of JCache.

This will ease the setup o very little clusters (2-3 nodes) in which any client in turn can act as coordinator

Comments from JIRA

Enrico Olivelli 2017-03-14T11:19:17.000+0100

added new value for blazingcache.mode=server

 

relevant configuration options:
{code:java}
String connect = properties_and_params.getProperty("blazingcache.zookeeper.connectstring", "localhost:1281");
int timeout = Integer.parseInt(properties_and_params.getProperty("blazingcache.zookeeper.sessiontimeout", "40000"));
String path = properties_and_params.getProperty("blazingcache.zookeeper.path", "/blazingcache");
boolean writeacls = Boolean.parseBoolean(properties_and_params.getProperty("blazingcache.zookeeper.writeacls", "false"));
String host = properties_and_params.getProperty("blazingcache.server.host", "");
int port = Integer.parseInt(properties_and_params.getProperty("blazingcache.server.port", "7000"));
boolean ssl = Boolean.parseBoolean(properties_and_params.getProperty("blazingcache.server.ssl", "false"));{code}
 

 

 

Enrico Olivelli 2017-03-14T11:26:17.000+0100

updated the docs at

https://dash.readme.io/legacy/project/blazingcache/v1.0/docs/getting-started

Enable Netty 4 native Epoll Linux support and upgrade to Netty 4.1.9Final

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-60

Reporter: eolivelli @eolivelli

Comments from JIRA

Enrico Olivelli 2017-03-14T10:00:17.000+0100

it will be enabled only if
{code:java}
 private static final boolean ENABLE_EPOLL_NATIVE = System.getProperty("os.name").equalsIgnoreCase("linux")
        && !Boolean.getBoolean("blazingcache.network.disablenativeepoll");{code}
So set system property blazingcaceh.network.disablenativeepoll to true to disable native epool

Implement SASL/Kerberos Authentication

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-47

Reporter: eolivelli @eolivelli

This is the backport of https://dev.majordodo.org/jira/browse/MAJ-69.

JAAS entries are:
BlazingCacheServer and BlazingCacheClient

Comments from JIRA

Enrico Olivelli 2016-10-18T17:24:33.000+0200

Example jass.conf for Kerberos:

{code}

BlazingCacheServer {
com.sun.security.auth.module.Krb5LoginModule required debug=true
useKeyTab=true
keyTab="/....file.keytab"
storeKey=true
useTicketCache=false
principal="blazingcache/[email protected]";
};

BlazingCacheClient {
com.sun.security.auth.module.Krb5LoginModule required debug=true
useKeyTab=true
keyTab="/...file.keytab"
storeKey=true
useTicketCache=false
principal="magnews/[email protected]";
};

{code}

Enrico Olivelli 2016-10-18T17:25:10.000+0200

Example configuratio for MD5
{code}

BlazingCacheServer {
org.apache.zookeeper.server.auth.DigestLoginModule required
user_hd="testpwd";
};

BlazingCacheClient {
org.apache.zookeeper.server.auth.DigestLoginModule required
username="hd"
password="testpwd";
};
{code}

JSR107 tck tests fail on master branch

Every tests failed with this error:
Expecting no open client connections. Customizations implementing Closeable need to be closed. See https://github.com/jsr107/jsr107tck/issues/100

Inefficient usage of Logger calls toString on any fetch request

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-69

Reporter: eolivelli @eolivelli

{code:java}
java.lang.String.valueOf(String.java:2994)
java.lang.StringBuilder.append(StringBuilder.java:131)
java.util.AbstractMap.toString(AbstractMap.java:559)
java.lang.String.valueOf(String.java:2994)
java.lang.StringBuilder.append(StringBuilder.java:131)
blazingcache.network.Message.toString(Message.java:156)
java.lang.String.valueOf(String.java:2994)
java.lang.StringBuilder.append(StringBuilder.java:131)
blazingcache.client.CacheClient.fetch(CacheClient.java:760)
blazingcache.client.CacheClient.fetch(CacheClient.java:715)
blazingcache.client.CacheClient.fetchObject(CacheClient.java:1377)
....
{code}

Comments from JIRA

Enrico Olivelli 2017-06-13T14:29:03.000+0200

I am marking this as "Higher" priority because it impacts on performance under heavy load (huge amount of fetches due to empty cache due to disconnection due to GC problems)

Enrico Olivelli 2017-07-25T12:57:11.000+0200

+1 LGTM

Batch local eviction notifications from client to server

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-45

Reporter: eolivelli @eolivelli

In 1.9.xx we are sending one eviction notification message for every entry to be locally evicted.
This can cause a lot of network traffic, and maybe saturate the TCP connection.

We can batch this notification messages in order to reduce the overhead

Comments from JIRA

Enrico Olivelli 2017-03-14T11:55:04.000+0100

Introduced new client-side parameter evictionBatchSize, which defaults to 100.

I think no one will ever change this value

Enrico Olivelli 2017-03-14T11:55:33.000+0100

The change is fully compatible with legacy pre-1.12 clients

Invalidations could be lost during network outage

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-34

Reporter: eolivelli @eolivelli

It happened that during a network problem the invalidate exited with a RuntimeException, instead of doing the usually "retry forever" loop.

The cause is that the Channel was "not valid" but still assigned to the cache client.
The retry login handles only "TimeoutException" and not other RuntimeException errors

Comments from JIRA

Enrico Olivelli 2016-05-11T16:19:02.000+0200

#59

New function CacheClient.load to load an entry to the local cache without remote update

JIRA: https://dev.blazingcache.org/jira/browse/BLAZ-46

Reporter: eolivelli @eolivelli

The idea is to let a client cache an entry locally without the need to update all the other peers with the same entry.

In fact if you set fetchPriorty=0 on some client A and client B fetches the entry it results in a cache "miss", this usually leads to a local local of the datum and a subsequent put, which in turn need to notify the client A.
Usually client A will be very "slow", because it has fetchPriority = 0 and so the "put" will be slow too

We can introduce a "load" operation, which loads the datum locally and registers the listener on server

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.