• bug report? yes
• feature request? no
• version: 2.5.6 (nette/caching)

Description

When trying to load cache by fallback in another fallback of load, script runs to infinity (maybe deadlock?)

Steps To Reproduce

public function runMethodWithCache() {
$cachedValue1 = $this->cache->load('cached_value_1', function(& $dependencies) {
        $dependencies['expire'] = '+1 day';
        $value = $this->runAnotherMethodWithCache();        
        return $value;
      });
}

private function runAnotherMethodWithCache() {
      $usedValue = 'something';
      $this->someValue = $this->cache->load('cached_value_2', function (& $dependencies) use ($usedValue) {
        $dependencies[Cache::ITEMS] = ['cached_value_1'];
        return $usedValue;
      });

     return 'something another';
}

When I comment following lines from second method, script running as expected.

      $this->someValue = $this->cache->load('cached_value_2', function (& $dependencies) use ($usedValue) {
        $dependencies[Cache::ITEMS] = ['cached_value_1'];
        return $usedValue;
      });

[UPDATE 1]: most probably, this is caused by cross dependency of Cache::ITEMS in second method, which is dependent on first one cache.

Idea

If subloading of cache is not possible, it should be inspect to avoid sub-loading of cache.

Look at the following code:

$output = $cache->load($cacheIdentifier, function() use ($view) {
  return $view->render();
});

This works nicely, but I can not define dependencies for this cached data when using the callback function.

Instead I have to write this:

$output = $cache->load($cacheIdentifier);
if (is_null($output)) {
  $output = $view->render();
  $cache->save($cacheIdentifier, $output, [
      Cache::TAGS => ['user/' . $userid]
  ]);
}

I'd like to see $dependencies as third optional parameter of load(). What do you think?

• bug report - NO
• feature request - YES
• version: >2.4.0

What's about

A few days ago I was working on cache panel. My goal was to show all created cache files with expiration, size and many other things. It was quite easy, let say with little help of reflection.

My result look's like that.

The thing is I cannot show cache key, because it's serialized in Cache object and passed to IStorage. I was reading it from FileStorage (that was my 1st implementation) and there is only md5 hash. I known I can inherit Cache object or create CacheFactory a everything gonna be allright, but I wont. I would like to see it also in my old projects.

Why it might be useful? At this time we have no cache tools or any other cache introspective tools. We can only create Cache object with IStorage and hope there's a cache file with this key (hash).

How to implement it

1) CacheFactory + inherit Cache object

It's good but only for new projects.

2) Store cache key under special tag

I think it's oukey, but little bit tricky.

$dependencies[Cache::TAGS] = ['__nette.cache.key' => $key];
$this->storage->write($key, $data, $dependencies);

3) Introduce new constant Cache::KEY

I consider this as best solution. It's clean and it shouldn't broke any existing code.

$dependencies[Cache::KEY] = $key;
$this->storage->write($key, $data, $dependencies);

What do you think guys? @dg @fprochazka @TomasVotruba @matej21 @JanTvrdik

Does it planned to implement?

Hello,

I use Nette 2.2.7. Today my production project totally crashed and doesn't work after error "Malformed journal file 'btfj.dat'".

My btfj.dat has 4 382 720 bytes after that error. In this file, there was no MAGIC header.

Here is first 5 000 bytes of broken btfj.dat: http://nette.nahledwebu.cz/btfj-error.dat
Here is first 5 000 bytes of btfj.dat, when is OK: http://nette.nahledwebu.cz/btfj-error.dat

This bug occurs probably after cleanTags() calls.

I've made dirty workaround in our project. After that exception catched, I delete btfj.dat, call clean(Cache:ALL) and send an e-mail to admin about that.

This bug is very dangerous - when occurs, whole website is broken and is required to manually fix it.

Following code leads to infinite wait on file lock here. Maybe there should be some global Exception handler with unlock?

$fn = function () use ($cache) {
    $cache->load('key', function () {
        throw new Exception;
    });
};

try {
    $fn();
} catch (Exception $e) {
    $fn();
}

Forum: http://forum.nette.org/en/21286-undefined-index-i-in-filejournal-php
Related issue: nette/nette#1076

I have a long-running script that runs in background and I use FileStorage::lock() to prevent multiple instances of the script to be running at the same time.

But there is a problem when the garbage collector runs the FileStorage::readMetaAndLock() method on the lock file, because it takes a long time to be released. The current process is suspended, until the long-running script releases the lock.

Please change class names of storages, because:

• MemcacheStorage doesn't exist
• MemcachedStorage doesn't support memcached extension
• NewMemcachedStorage... Shame on the author of this class name

Thanks for your time, I hope that you will change it and I will never find anything like NewNewMemchachedStorage after another memory cache extension will be released.

Newly added optimizations of SQLiteJournal caused problem for using SQLiteJournal for apps running in virtual machine (tested in Virtualbox). WAL option doesn't work with network filesystem which is used in Virtualbox.

cause:

PRAGMA journal_mode = WAL;

in SQLiteJournal.php:36

Way to switch WAL on/off in journal or removal of the line would fix the problem.

Environment:
Windows 10 x64
Container running in Docker Toolbox v1.9.1f
PHP 5.6.16

Sometimes I need debug application performance and I want detect how many time and how many requests cache must do.

Imagine something like this panel:

I think logging process should be implemented in Storage like low-level operation and specific Storage should implement logging proces.

In specific Storage can be array like this:

private $log = [
   [ // sample record. By default will be empty array
      'action' => 'read',
      'key' => 'article-list',
      'params' => [],
      'time' => 24.7
   ]
];

public function getLog(): array {
   return $this->log;
}

In Tracy I can read current IStorage and get list of actions by optional public getter.

I can implement it for native package storages.

What do you think? Thanks.

When I used "nette/nette": "~2.3", in my project it installed nette/caching as dependency on this revision a89c504

And when i tried to run project there was exception: PDOException: could not find driver,
with path to temp/cache/journal.s3db (file really not exists)

After this command everything work ok:
vendor/nette/caching$ (master) git reset --hard HEAD~1
HEAD is now at 47bd309 typo in class name

PHP 5.6.4-4ubuntu6
Tracy 2.3.2
Server Apache/2.4.10 (Ubuntu)
Nette Framework 2.3-dev

If i want implement apcu by IStorage then apcu has own atomic method, whose works fine. But interface of IStorage does not allow use callback for apcu_entry.

For this moment locking solves cache and i can't to use native lock from strorage. Same problem is with redis where in implementation is useful callback.

I understand this is probably BC break. I try to create implementation where i add new interfaces ISelfLock, ISimpleStorage and IStorage has still same api.

I made little test where files blocking concurrent processes while apcu with ISelfLock does not do it.

Here is other info (in czech)

Version: 3.2.2

Bug Description

https://forum.nette.org/cs/36236-cache-invalidace-podle-patternu#p225445

$cache1 = new Cache($this->cacheStorage, 1);
$cache2 = new Cache($this->cacheStorage, 2);

$cache1->save("key1-1", "value-1-1", [Cache::Priority => 1]);

$cache2->save("key2-1", "value-2-1", [Cache::Priority => 1]);
$cache1->clean([
    Cache::Priority => 1
]);

Assert::null($cache1->load("key1-1"));
Assert::notNull($cache2->load("key2-1")); // Tester\AssertException: Value should not be null

Expected Behavior

Cache1 should not be involved in cleaning of Cache2 - since they have different namespace.

Possible Solution

Problem lies when both caches shares same Storage. Storage returns simply keys without respect to namespace. Namespace should probably be stored in cache storage and cleaning function should pay respect to that namespace.

Hello, I needed to create PSR16 like implementation for Nette.

Could I add to this repository as a bridge?

I can prepare PR, if you agree.

8bf7455

Hi, in version 3.1.3, constant NAMESPACE_SEPARATOR disappeared and is no longer available. And also backward compatible name has not been added. Can you please add upper snake case constant?

Also, why Pascal Case for class constants? This is opposed to PSR-12, which you claim, that Nette follows: https://doc.nette.org/en/contributing/coding-standard. PSR-12 states that PSR-1 must be adhered. And PSR-1 states that "Class constants MUST be declared in all upper case with underscore separators". So why is Nette not following PSR-12 coding standard?

EDIT: also, why is backward-incompatible change released as patch version and not major version?

Thanks.

To be consistent with FileJournal, SQLite database should have 0666 - umask() permissions.

#28 (comment)

Applications can be very complex and not all developers use the maximum cache expiration time correctly, which causes us to accumulate data on disk. It would be good to implement a native way to set a global maximum cache expiration time.

A particular application would then set the expiration to, for example, 3 days, and the server administrator would always know that even if the developer sets the caching to a longer period, it will be reduced to the maximum period.

Thank you.

I've switched our project to Nette\Caching v2.4 due to another bug (issue #18).

But today, an application has failed with PDOException #HY000 SQLSTATE[HY000]: General error: 1 too many SQL variables.

I can't provide error log in RAW HTML output (proprietaly real application), but I can part of Stack Trace as image:

Latte create chache dir named 'latte', but tag ALL delete dirs/files only if start with _.
For that reason it never delete latte cache.
https://github.com/nette/caching/blob/master/src/Caching/Storages/FileStorage.php#L270

• I found out Memcached is not storing some structure I wanted. I discovered that set() function fails, responding with false. Later I discovered reason using Memcached::getResultCode() which I found in phpDoc.

• This response, however is not handled in Nette. I suggest failure on write to cache should at least trigger notice for user.

I will gladly make a PR, but before I would like to know author's opinion.

Instead of

I would suggest something like:

		$stored = $this->memcached->set($key, $meta, $expire);
                
                if(!$stored){
                    trigger_error("Storing to memcached failed with error ".$this->memcached->getResultCode().": " . $this->memcached->getResultMessage(), E_USER_NOTICE);
                }

⚠️ Adding new Notice might consider that as a Breaking Change - Im not completely familiar with these internal rules.

Is redis not supported?
It would be better if we could support redis

On PHP 5.5.9 (maybe on others too, I did not tested), when sqlite extension is missing, nette throws misleading error:

Nette\InvalidStateException
CacheJournal has not been provided

I had v2.4.4 of nette/caching installed.

Maybe caching should test if sqlite extension is installed and throw proper exception if not.

Couple of times per month I receive unserialize error ("Error at offset X of Y bytes") from RobotLoader which uses FileStorage. This suggests that FileStorage failed to write tha data properly. I alway delete the cache which solves the issue, obviously, but it is getting quite annoying so I've decided to finally open this issue.

Windows 10 x64, PHP 7.0.8 x64 NTS, FastCGI + Caddy

In release announced for version 2.4.5, title is for old version.

I know this is rather server problem, but how do you handle problems with reusing an existing connection which is made persistent in SQLiteJournal storage by default? I'm facing random errors like
SQLSTATE[HY000]: General error: 1 cannot start a transaction within a transaction or SQLSTATE[HY000]: General error: 10 disk I/O error (mainly on production server which is PHP 5.4.41-0+deb7u1, nginx/1.2.1). The only thing which helps me is switching back to FileJournal.

In a high-concurrency system, at the moment of cache failure, a large number of requests will enter the database, which can easily cause the database to crash

reference：
https://www.wikiwand.com/en/Cache_stampede
https://symfony.com/doc/current/components/cache.html

Version: 3.x-dev

Bug Description

unserialize(): Error at offset 0 of 145 bytes (variable number of bytes)

File: ...\nette\caching\src\Caching\Storages\FileStorage.php:333

$variable_representation | "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00a:2:{s:2:"id";b:1;s:4:"code";b:1;}" (127)

Steps To Reproduce

Problem is with PHP 7.3 (rc3) on some pages only. After refresh page there is this problem. Cache is deleted. It seems that data is from Sessions - problem start on with this code:
$this->currency = $this->facade->getById($this->sessionSection->currency['id']);

Facade:
$data = $this->db->table($this->repository)->wherePrimary($id)->fetch();

Stack trace:
2) ...\nette\caching\src\Caching\Storages\FileStorage.php:85

$meta | array (4)
time => "0.78053500 1540276060"
serialized => true
file => "../temp/cache/_Nette.Database.83a928a729900849010020b112a30316/_6034dbe4a271e54806a0513243994932"
handle => unknown type
-- | --

Content of file: 000068a:2:{s:4:"time";s:21:"0.09111900 1540276063";s:10:"serialized";b:1;} a:0:{}

3. ...\vendor\nette\caching\src\Caching\Cache.php
4. ...\database\src\Database\Table\ColumnAccessCache.php:128

Possible Solution

There is maybe dependency with solution of nette/database#208 (comment)

Moved from nette/nette#592

Bug Description

SQLliteJournal is slow. See also: https://forum.nette.org/cs/24571-nette-caching-sqllitejournal-velmi-pomaly
Bigger Journal file under heavier load of writes and reads of cache Journal may end in Database Locked error.

I propose adding a new interface IMultiReadStorage that will extend Nette\Caching\IStorage and it's implementation would mean that the cache can utilize the storage more effectively. Because reading 100 keys from memory cache at once is more effective than 100 requests.

Yes, this is a real world use-case, tested in our application and a friend on Posobota also told me he's using this approach.

We did implement our own IMultiReadStorage to Kdyby/Redis, but I wanna have a "standard" interface for my storage.

Also to the cache class should be added new method

class Cache
{
    public function multiLoad(array $keys)
    {
        if ($this->storage instanceof IMultiReadStorage) {
            return $this->storage->multiRead($keys);
        }

        $result = array();
        foreach ($keys as $key) {
            $result[$key] = $this->load($key);
        }
        return $result;
    }

To discuss:

• should this be in Nette? (Should I send a pullrequest?)
• better name for IMultiReadStorage ?
• implementation details