amphp / mysql Goto Github PK
View Code? Open in Web Editor NEWAn async MySQL client for PHP, optimizing database interactions with efficient non-blocking capabilities. Perfect for responsive, high-performance applications.
License: Other
An async MySQL client for PHP, optimizing database interactions with efficient non-blocking capabilities. Perfect for responsive, high-performance applications.
License: Other
ConnectionPool creates spare connections, even if I don't use a single one.
If 0 connections are used, at least 1 is created
If 1 conneciton is used, 3 are created
If addConnection
in c'tor and
if (count($this->ready) < 2) {
$this->addConnection();
}
are removed, the number of connection seems to be correct.
Using 21801b0 Currently a connection pool owned by an Aerys application is not able to recover when the database restarts. Error:
"ArgumentCountError: Too few arguments to function Amp\\Mysql\\Connection::Amp\\Mysql\\{closure}(), 0 passed in ./vendor/vendor/amphp/mysql/lib/Stmt.php on line 42 and exactly 1 expected in ./vendor/vendor/amphp/mysql/lib/Connection.php:40
Stack trace:
#0 ./vendor/vendor/amphp/mysql/lib/Stmt.php(42): Amp\\Mysql\\Connection::Amp\\Mysql\\{closure}()
#1 ./vendor/vendor/amphp/mysql/lib/Stmt.php(139): Amp\\Mysql\\Stmt->conn()
#2 ./vendor/src/DataStorage/MysqlAsyncStorage.php(228): Amp\\Mysql\\Stmt->execute(Array)
#3 ./vendor/src/DataStorage/MysqlAsyncStorage.php(131): Fs\\Blacklist\\DataStorage\\MysqlAsyncStorage->do('msisdn_permissi...', Array)
#4 ./vendor/src/RouteHandler/V3/MsisdnStatusHandler.php(53): Fs\\Blacklist\\DataStorage\\MysqlAsyncStorage->getMsisdnPermissionHistory('vivo', '5511981349170', NULL)
#5 ./vendor/src/AbstractRestHandler.php(73): Fs\\Blacklist\\RouteHandler\\V3\\MsisdnStatusHandler->get(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse), Object(stdClass))
#6 ./vendor/vendor/amphp/aerys/lib/Bootstrapper.php(212): Fs\\Blacklist\\AbstractRestHandler->__invoke(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse), Array)
#7 [internal function]: class@anonymous->__invoke(Object(Aerys\\StandardRequest), Object(Aerys\\StandardResponse))
#8 ./vendor/vendor/amphp/amp/lib/functions.php(791): Generator->current()
#9 ./vendor/vendor/amphp/amp/lib/functions.php(779): Amp\\__coroutineAdvance(Object(Amp\\CoroutineState))
#10 ./vendor/vendor/amphp/aerys/lib/Server.php(765): Amp\\resolve(Object(Generator))
#11 ./vendor/vendor/amphp/aerys/lib/Server.php(716): Aerys\\Server->tryApplication(Object(Aerys\\InternalRequest), Object(class@anonymous), Array)
#12 ./vendor/vendor/amphp/aerys/lib/Server.php(555): Aerys\\Server->respond(Object(Aerys\\InternalRequest))
#13 ./vendor/vendor/amphp/aerys/lib/Server.php(530): Aerys\\Server->onParsedMessageWithoutEntity(Object(Aerys\\Client), Array)
#14 ./vendor/vendor/amphp/aerys/lib/Http1Driver.php(318): Aerys\\Server->onParseEmit(Object(Aerys\\Client), 2, Array, NULL)
#15 [internal function]: Aerys\\Http1Driver->parser(Object(Aerys\\Client))
#16 ./vendor/vendor/amphp/aerys/lib/Server.php(524): Generator->send('GET /v3/vivo/ms...')
#17 ./vendor/vendor/amphp/amp/lib/UvReactor.php(424): Aerys\\Server->onReadable('000000004153540...', Resource id #268, Object(Aerys\\Client))
#18 ./vendor/vendor/amphp/amp/lib/UvReactor.php(401): Amp\\UvReactor->invokePollWatcher(Object(stdClass))
#19 [internal function]: Amp\\UvReactor->Amp\\{closure}(Resource id #269, 0, 1, Resource id #268)
#20 ./vendor/vendor/amphp/amp/lib/UvReactor.php(93): uv_run(Resource id #60, 1)
#21 ./vendor/vendor/amphp/amp/lib/functions.php(46): Amp\\UvReactor->run(Object(Closure))
#22 ./vendor/vendor/amphp/aerys/bin/aerys(106): Amp\\run(Object(Closure))
#23 {main}
I am using getConnection for my transaction.
$mysqlPool = new Mysql\Pool("host=".DB_HOST.";user=".DB_USER.";pass=".DB_PWD.";db=".DB_NAME, CONN_LIMIT);
var_dump($mysqlPool->getConnection());
Result:
NULL
I expect returning a promise, but null is returned.
Uncaught Error: Call to a member function isReady() on null in /vendor/amphp/mysql/lib/ConnectionPool.php:136
This error happens fairly often.
$pool = new Pool();
$promises[] = $pool->query('<query>');
$promises[] = $pool->query('<query>');
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query('<query>'));
$promises[] = $pool->query'<query>'));
foreach ($promises as $promise) {
// Retrieve the result set
\Amp\wait($promise);
}
#21 {main}
[2017-05-31 14:00:38] lumen.ERROR: Symfony\Component\Debug\Exception\FatalThrowableError: Call to a member function succeed() on null in /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php:458
Stack trace:
#0 /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php(1086): Amp\Mysql\Processor->handleOk('\x00\x00\x00\n\x00\x00\x00')
#1 /home/vagrant/fras/source/lumen/vendor/amphp/mysql/lib/Processor.php(926): Amp\Mysql\Processor->parsePayload('\x00\x00\x00\n\x00\x00\x00')
#2 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(265): Amp\Mysql\Processor->onRead('000000003d7266f...', Resource id #511, NULL)
#3 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(249): Amp\NativeReactor->doIoCallback('000000003d7266f...', Object(stdClass), Resource id #511)
#4 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(214): Amp\NativeReactor->selectActionableStreams(0.5947)
#5 /home/vagrant/fras/source/lumen/vendor/amphp/amp/lib/NativeReactor.php(156): Amp\NativeReactor->doTick(false)
There should be a mechanism like in \Amp\Socket\BasicSocketPool that closes idle connection after a defined amount of time.
Currently it is not possible to release idle connections gracefully. You can only dispense / replace the whole pool to get rid of idle connections which IMHO is not a viable solution.
In production peaks can created. E.g. by a slightly overloaded server. Not releasing idle connection can create a failure cascade e.g. when the server reaches max connections and starts dropping connection attempts.
This is unlikely to be an issue with a hand full of clients but if say 32 aerys workers are running on 30 machines each, each of them creating 10, 20, 50 connections to the same server AND not releasing them - this could become an issue.
I'm attempting to run example 003_generic_with_yield inside a vagrant box. The guest OS is Centos 6.4, the host is OSX.
Using a valid IP address the code appears to cause a buffer overflow. This happens whether I use the IP address of '10.0.2.2' which is the correct one that has MySQL running on it, or '127.0.0.1' which obviously exists, but doesn't have MySQL running on it.
There error isn't shown if I attempt to use an IP address that is unreachable.
[root@rpm examples]# strace /usr/local/bin/php 003_generic_with_yield.php > AmpMySQLOverflow.txt 2>&1
*** buffer overflow detected ***: strace terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7fdb95bb7507]
/lib64/libc.so.6(+0x1003f0)[0x7fdb95bb53f0]
/lib64/libc.so.6(+0xff2a7)[0x7fdb95bb42a7]
strace[0x409c1c]
strace[0x40592f]
strace[0x403045]
strace[0x404661]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x7fdb95ad3cdd]
strace[0x401fd9]
======= Memory map: ========
00400000-0044a000 r-xp 00000000 fd:00 4460201 /usr/bin/strace
0064a000-0064b000 rw-p 0004a000 fd:00 4460201 /usr/bin/strace
0064b000-00659000 rw-p 00000000 00:00 0
00671000-00692000 rw-p 00000000 00:00 0 [heap]
7fdb9589f000-7fdb958b5000 r-xp 00000000 fd:00 28836039 /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb958b5000-7fdb95ab4000 ---p 00016000 fd:00 28836039 /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb95ab4000-7fdb95ab5000 rw-p 00015000 fd:00 28836039 /lib64/libgcc_s-4.4.7-20120601.so.1
7fdb95ab5000-7fdb95c3f000 r-xp 00000000 fd:00 28835854 /lib64/libc-2.12.so
7fdb95c3f000-7fdb95e3e000 ---p 0018a000 fd:00 28835854 /lib64/libc-2.12.so
7fdb95e3e000-7fdb95e42000 r--p 00189000 fd:00 28835854 /lib64/libc-2.12.so
7fdb95e42000-7fdb95e43000 rw-p 0018d000 fd:00 28835854 /lib64/libc-2.12.so
7fdb95e43000-7fdb95e48000 rw-p 00000000 00:00 0
7fdb95e48000-7fdb95e68000 r-xp 00000000 fd:00 28835847 /lib64/ld-2.12.so
7fdb9605b000-7fdb9605e000 rw-p 00000000 00:00 0
7fdb96065000-7fdb96067000 rw-p 00000000 00:00 0
7fdb96067000-7fdb96068000 r--p 0001f000 fd:00 28835847 /lib64/ld-2.12.so
7fdb96068000-7fdb96069000 rw-p 00020000 fd:00 28835847 /lib64/ld-2.12.so
7fdb96069000-7fdb9606a000 rw-p 00000000 00:00 0
7fffcbe85000-7fffcbe9a000 rw-p 00000000 00:00 0 [stack]
7fffcbf03000-7fffcbf04000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
The full output of the strace is too large to post here; it is on S3:
https://s3.amazonaws.com/static.basereality.com/AmpMySQLOverflow.txt
mysql has a setting to kill idle connections after a while: wait_timeout - https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_wait_timeout - with a default of 8 hours.
This regularly causes exceptions on very low-frequency ends, while it would be easily avoidable if we restarted connections from time to time.
We might desire to request that timeout from the server via SHOW VARIABLES LIKE 'wait_timeout'
on pool startup and restart (or just kill them as long as more than 2 connections are available) the idling connections if their last use gets dangerously close to wait_timeout.
I think it would be good to use URIs to configure connections, e.g. mysql://user:password@host:port/database?timeout=abc
. URIs provide everything we need: User + password on host / port, a database name as path and options via URL parameters.
// 001_connect.php
Amp\run(function() {
$db = new \Mysql\Pool(DSN);
// getConfig() added to my source: return Pool::$config;
echo $db->getConfig()->charset , "\n";
$db->setCharset("latin1_general_ci");
echo $db->getConfig()->charset , "\n";
$db->close();
});
// php 001_connect.php
awkwardly it runs forever.
If I yield $db->setCharset("latin1_general_ci");
it's runs, but I still get
an exception with message 'Connection not ready, cannot send any packets' in
Connection.php:1647'.
In other way if I yield $db->query("SET NAMES 'latin1' COLLATE 'latin1_general_ci'");
(replacing setCharset()
) it runs flawlessly.
Looks like Pool::$connections
is never filled upon __construct
, only if you use
$db = (yield new Mysql\Pool(DSN));
then Pool::$connectios
has 1 item (as expected?),
but still can't use $db->setCharset
directly, I needed to
yield $db->setCharset("latin1_general_ci");
.
// finally...
<?php
require './example_bootstrap.php';
Amp\run(function() {
$db = (yield new Mysql\Pool(DSN));
echo $db->getConfig()->charset , "\n"; // utf8mb4
yield $db->setCharset("latin1_general_ci");
echo $db->getConfig()->charset , "\n"; // latin1
$db->close();
});
Thoughts?
$ php -v
PHP 5.6.5-1 (cli) (built: Jan 27 2015 15:59:26)
Using latest master, calling Connection::close always triggers a warning:
fclose(): supplied resource is not a valid stream resource in ../vendor/amphp/mysql/lib/Connection.php:1072
Debugger shows that $connection->close()
is being called twice but turns out in user space domain I'm only calling ->close()
once:
class MyAsyncClientClass
{
private
$pool
;
function __construct($pool)
{
$this->pool = $pool;
}
function transaction(\Closure $fn)
{
$connection = yield $this->pool->getConnection();
try {
yield $connection->query('START TRANSACTION;');
$result = yield from $fn($connection);
yield $connection->query('COMMIT;');
return $result;
} catch (\Exception $e) {
$connection->query('ROLLBACK;');
throw $e;
}
finally {
$connection->close(); // always triggers fclose() warning
}
}
}
If close is supposed to be called implicitly even when the connection was acquired, this may not be a bug. Anyway it would be nice to know when the Pool will close the connection even when the acquired connection was unmapped.
If close is supposed to do not be called implicitly when the connection was acquired, this may be a bug.
I've tried the following patch and, even though it makes sense, it caused connections to never be closed:
private function closeSocket() {
$this->disableRead();
$this->disableWrite();
+ if (is_resource($this->socket)) {
- @fclose($this->socket);
+ fclose($this->socket);
+ }
$this->connectionState = self::CLOSED;
}
If a connection isn't used for a long time, it still remains open. If the next query is issued a long time after then, it fails, because the connection went away. This shouldn't happen, it should be handled automatically internally.
I am connecting to an amazon DB from windows php 7.2.4 with following code:
https://gist.github.com/plankes-projects/d941f49bf74353f0b29700261192c639
I get DNS timeout exception when I am running it on my windows developer machine.
https://gist.github.com/plankes-projects/c3cfd2a6aead806adc7a9efaf545ba48
Everthing works fine if I am using this config with eloquent or if I am connecting to a local DB:
https://gist.github.com/plankes-projects/afbe1d66eb08262a0f1484cf9fbaf402
Following amphp versions are installed with composer:
amphp/amp dev-master 9b2fb76 A non-blocking concurrency framework for PHP applications.
amphp/byte-stream v1.3.1 A stream abstraction to make working with non-blocking I/O...
amphp/cache v1.2.0 A promise-aware caching API for Amp.
amphp/dns v0.9.12 Async DNS resolution for Amp.
amphp/file v0.3.1 Allows non-blocking access to the filesystem for Amp.
amphp/mysql dev-master aec58f4 Asynchronous parallel Mysql client built on the Amp concur...
amphp/parallel v0.2.5 Parallel processing component for Amp.
amphp/parser v1.0.0 A generator parser to make streaming parsers simple.
amphp/process v0.3.3 Asynchronous process manager.
amphp/socket v0.10.8 Async socket connection / server tools for Amp.
amphp/sync v1.0.1 Mutex, Semaphore, and other synchronization tools for Amp.
amphp/uri v0.1.3 Uri Parser and Resolver.
amphp/windows-registry v0.3.1 Windows Registry Reader.
Hi,
Since I raised this concern in stack overflow as well but couldn't find the answer, I have put it here: https://stackoverflow.com/questions/52058338/amphp-mysql-not-returning-all-the-records
We have started using amphp\mysql to run pool of connections for threading concept. We have a table "content" which has 100k of records. Trying to list out for further processing but fetching only 42 records. Also, doesn't have any LIMIT statement.
What am I missing in my code?
Below is the code:
require "vendor/autoload.php";
use Amp\Loop;
use Amp\Artax\Request;
use Amp\Artax\Response;
$db = '';
Amp\Loop::run(function () {
$client = new Amp\Artax\DefaultClient;
$db = Amp\Mysql\pool("host=".$this->conn_prop['db_host']."; user=".$this->conn_prop['db_user']."; pass=".$this->conn_prop['db_passwd']."; db=".$this->conn_prop['db_name']);
$statement = (yield $db->query("SELECT registration_id,w_id,ip,device_type FROM content WHERE active_status='1' AND ip !=''"));
$cnt = 0;
while (yield $statement->advance()) {
echo $cnt++;
$row = $statement->getCurrent();
if(!empty($row)){
var_dump($row);
.................
................
}
}
});
Waiting for the suggestion.
The code here seems quite wrong.
It's not checking that $socket is not null, and so I'm getting the error fclose() expects parameter 1 to be resource, null given in ...amp-mysql/lib/Connection.php on line 138
.
Also, I think it should be checking for the error first and calling the $future->fail regardless of connection state.
To avoid bigger latencies when multiple prepared statements are prepared on a same connection and they're all executed simultaneously.
[And not just when connections crashed.]
$query = yield $db->prepare("SELECT ?", [2]);
var_dump(yield $query->fetchRow()); // OK : array ( 0 => 2 )
$query = yield $db->prepare("SELECT ?", [1.5]);
var_dump(yield $query->fetchRow()); // KO : array ( 0 => 3.13984E-319 )
Just try that: SELECT "" foo
If you will run following code:
$stmt->bind('id', [1,2,3,4,5]);
you will get following error:
Notice: Array to string conversion
As quick solution you can use Doctrine helper function to convert params to flat array:
list($sql, $params, $paramsType) = SQLParserUtils::expandListParameters($query->getSQL(), $query->getParameters(), $query->getParameterTypes());
Connection::parseConnStr($connStr, $sslOptions) fails with " Undefined offset: 1"
Connection string
limit=2;host=<host>;user=<user>;pass=<pass>;db=<database>
I have a table with 100000 rows with some test data. When i try to select all rows from table i have very large the memory usage. I thought that this value will not increase depending on the number of rows. Is this expected behavior or is it a memory leak?
My example:
$db = Amp\Mysql\pool("...");
$start = microtime(true);
/** @var \Amp\Mysql\ResultSet $result */
$result = yield $db->query("SELECT * FROM `amp_memory`;");
while (yield $result->advance()) {
// nothing
}
$end = microtime(true);
echo 'Selected 100000 rows: ' . ($end - $start) . PHP_EOL;
echo memory_get_peak_usage(true)/1024/1024 . ' MB' . PHP_EOL;
$db->close();
Output:
Selected 100000 rows: 2.8150889873505
192.00390625 MB
I got a connection from the pool for doing transaction.
How can I release it back to the pool after commit?
for the current design, it seems I need to close that connection myself. because the connection I got is removed from the connection pool.
It seems the development of this project is being inactive. But I have to say, this is an important project for making PHP better to compare with node.js. Thanks for amphp, I can code event-driven now. But the supporting library is so few. Cheer up. Make amphp better. yield
syntax is far much better than evil then
.
Can't you pass the error code to the QueryError like in
http://php.net/manual/de/mysqli.errno.php
Or maybe sth more interoperable with PgSql, like an QueryExecutionError?
When I am starting my unit test I get to this location which, if I step over it, produces a seg fault. Sadly I cannot share my unittest because I was not able to reproduce this in an easier environment. But I attached a screenshot of the stacktrace.
I think this is because I run into an infinity loop, visible in the screenshot.
The line of code I am breaking has jumped to Coroutine.php:93 on the previous 3 skips as visible in the screenshot.
I can reproduce this issue every time, so feel free to ask for more information.
amphp was installed with composer and has following versions:
amphp/amp dev-master c632e40 A non-blocking concurrency framework for PHP applications.
amphp/byte-stream v1.3.1 A stream abstraction to make working with non-blocking I/...
amphp/cache v1.2.0 A promise-aware caching API for Amp.
amphp/dns v0.9.12 Async DNS resolution for Amp.
amphp/file v0.3.1 Allows non-blocking access to the filesystem for Amp.
amphp/mysql dev-master f3e1f15 Asynchronous parallel Mysql client built on the Amp concu...
amphp/parallel v0.2.5 Parallel processing component for Amp.
amphp/parser v1.0.0 A generator parser to make streaming parsers simple.
amphp/process v0.3.2 Asynchronous process manager.
amphp/socket v0.10.8 Async socket connection / server tools for Amp.
amphp/sync v1.0.1 Mutex, Semaphore, and other synchronization tools for Amp.
amphp/uri v0.1.3 Uri Parser and Resolver.
amphp/windows-registry v0.3.1 Windows Registry Reader.
Looking at server logs the following notice happens quite frequently:
notice Uninitialized string offset: 9 in ../vendor/amphp/mysql/lib/DataTypes.php on line 202
Database interaction works fine, but this impedes usage of a more picky error handler.
After following warning process hung:
Warning: unpack(): Type V: not enough input, need 4, have 1 in ./vendor/amphp/mysql/lib/DataTypes.php on line 293
Currenlty I'm not able to find steps to reproduce. It appears randomly.
I am trying to get the last insert id. But I am using it in pool mode, will it affect the result by other inserts?
$resultSet = yield $pool->prepare("INSERT INTO `table` (`data`) VALUE (:data)", ['data' => $data]);
$insertId= $resultSet->getConnInfo()->insertId;
Is it the correct code to get the insertId?
@trowski
This issue is more a reminder or discussion than a perfectly investigated thesis.
I had in mind, you moved a lot of logic from Connection to Processor. From what I can see, the processor was meant to be a kind of state-machine to process server responses / connection states.
While moving connection logic (and socket stuff) into the processor its testability suffers.
With a processor that is not coupled to the IO method (connection, socket, whatever) it would be much easier to test the state machine and the protocol.
I really appreciate what you achieved with the mysql module (totally!) but I got the notion it is not really well tested or even thoroughly testable in edge cases with the current architecture.
Some of the edge cases I ran into are either not easily or not at all reliably testable with a real server instance - what you obviously to in your tests.
Using a module like that can be really mission critical for guys like me and if there is a problem with a release in can cause REAL trouble.
Edge case examples:
What do you think?
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- Installation request for amphp/mysql ^0.1.6 -> satisfiable by amphp/mysql[v0.1.6].
- amphp/mysql v0.1.6 requires amphp/amp ^1 -> no matching package found.
Potential causes:
- A typo in the package name
- The package is not available in a stable-enough version according to your minimum-stability setting
see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.
Installation failed, reverting ./composer.json to its original content.
It seems amphp/mysql is trying to require amphp/amp v1, but the latest version of amphp/amp is v2 now. when will it support to it?
Notice: Undefined index: 000000002f21b38f000000002e01fac2 in mysql/lib/Pool.php on line 242
There should be a check if that key exists.
https://github.com/valVk/amph-mysql
Here is in table is 565529 rows but in fact I can get only first 92 rows.
Snapshot of system in repository
First of all, thank you for a great library.
In Amp\Mysql\Internal\Processor->successfulResultsetFetch()
method, there can be a case where $this->result
is null
, but a method is executed on it:
private function successfulResultsetFetch() {
$deferred = &$this->result->next;
if ($this->connInfo->statusFlags & StatusFlags::SERVER_MORE_RESULTS_EXISTS) {
$this->parseCallback = [$this, "handleQuery"];
$this->addDeferred($deferred ?: $deferred = new Deferred);
} else {
if (!$deferred) {
$deferred = new Deferred;
}
$deferred->resolve();
$this->parseCallback = null;
$this->query = null;
$this->result = null;
$this->ready();
}
$this->result->updateState(ResultProxy::ROWS_FETCHED);
}
In Amp\Mysql\Internal\Processor->prepareFields()
there is a similar situation, but $this->result
is stored in the temporary variable. When I use this approach in the situtaion above, the code goes fine, until $db->close()
: "Connection went away... unable to fulfil this deferred ... It's unknown whether the query was executed..."
(somehow there are still some elements in the $this->deferreds
array during closing).
By the way I do not know why, but the script fails silently, and the only way I was able to see exception is to alter Amp\Internal\Placeholder
:
/**
* @param \Throwable $reason Failure reason.
*/
private function fail(\Throwable $reason) {
var_dump($reason->getMessage());
var_dump($reason->getTraceAsString());
$this->resolve(new Failure($reason));
}
Thank you in advance.
There should be examples for transactions and probably a dedicated API.
I made a fork with Amp v2 support.
https://github.com/optiman/mysql/tree/amp_v2
@bwoebi, if you create amp_v2
branch, I could make a pull request.
At the moment, amphp/mysql
provides fetchRow
, fetchObject
and fetch
, but no fetchAssoc
.
It's very common in mysqli
and PDO
. I recommend to add this method to provide string index result without combining with numeric index.
There should be a README
with some examples / introduction that also points to /examples
.
PHP deprecates each
and (unset)
, both are used in this package and should be removed.
I'm seeing a large perf impact on prepared queries vs regular queries :
\Amp\Promise\wait(
\Amp\call(function() {
$db = yield \Amp\Mysql\connect('host=127.0.0.1;user=root;pass=foobar;db=my_db');
for ($i = 0; $i < 10; $i++) {
$time = microtime(true);
$results = yield $db->execute("SELECT :rand", [ 'rand' => rand() ]);
yield $results->advance();
echo round((microtime(true) - $time) * 1000, 2) . " ms" . PHP_EOL;
}
})
);
outputs :
2.73 ms
40.8 ms
39.99 ms
40.22 ms
40.02 ms
39.91 ms
40 ms
39.98 ms
40 ms
39.57 ms
Same code using a regular query :
\Amp\Promise\wait(
\Amp\call(function() {
$db = yield \Amp\Mysql\connect('host=127.0.0.1;user=root;pass=foobar;db=my_db');
for ($i = 0; $i < 10; $i++) {
$time = microtime(true);
$results = yield $db->query(sprintf("SELECT %d", rand()));
yield $results->advance();
echo round((microtime(true) - $time) * 1000, 2) . " ms" . PHP_EOL;
}
})
);
outputs :
1.89 ms
0.37 ms
0.37 ms
0.36 ms
0.36 ms
0.36 ms
0.36 ms
0.36 ms
0.41 ms
0.32 ms
40ms overhead per query seems fairly taxing for the confort of prepared queries ?
When i try to insert json
value in my table
Amp\Loop::run(function () {
$db = Amp\Mysql\pool("...");
yield $db->query("CREATE TABLE IF NOT EXISTS amp_json (a INT(10), b JSON)");
/** @var \Amp\Mysql\Statement $statement */
$statement = yield $db->prepare("INSERT INTO amp_json (a, b) VALUES (?, ?)");
foreach (range(1, 10) as $num) {
$json = json_encode(['num' => $num]);
yield $statement->execute([$num, $json]);
}
$db->close();
});
I get an error:
Fatal error: Uncaught Amp\Mysql\QueryError: MySQL error (3144): #22032 Cannot create a JSON value from a string with CHARACTER SET 'binary'. Current query was INSERT INTO amp_json (a, b) VALUES (?, ?) in /vendor/amphp/mysql/src/Internal/Processor.php:617
I was able to insert a row with json
value only by concatenating, but it's not good solution.
yield $db->execute("INSERT INTO amp_json (a, b) VALUES ($num, '" . json_encode(['num'=>$num]) . "')");
May be you have some ideas about it?
This notice happens when querying insert statements or calling $pool->ping()
. Looks like a minor bug as database interactions still works as expected but it's a bummer when using a more strict error handler that transforms all levels of errors into exceptions:
set_error_handler(function($errno, $errstr, $file, $line) {
throw new \Exception("{$errstr} in {$file}:{$line}", $errno);
});
Error is:
notice Uninitialized string offset: 7 in /.../vendor/amphp/mysql/lib/DataTypes.php on line 202
When a server goes away (which is ok when wait_timeout is set), there must be a way to retry the query with a different connection.
I implemented that outside of amp/mysql but this leads to unresolved query promises in many cases.
See script to reproduce:
https://gist.github.com/brstgt/f7b23ef4a174e950587e0aed4e0bd245
Output looks like:
Try
Try
MySQL went away on query SELECT ...
Try
Got response
[Script ends]
Should look like
Try
Try
MySQL went away on query SELECT ...
Try
Got response
Resolve
DONE
After some benchmarking I have figure out that zlib compresion affects query performance a lot.
w/ zlib:
+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+
| benchmark | subject | groups | params | revs | its | mem_peak | best | mean | mode | worst | stdev | rstdev | diff |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+
| QueryBench | benchPdoQueries | | [] | 1 | 10 | 4,439,176b | 1.17000ms | 1.87690ms | 1.68573ms | 3.83200ms | 0.68067ms | 36.27% | 0.00% |
| QueryBench | benchSyncQueries | | [] | 1 | 10 | 4,439,184b | 5.76100ms | 46.13990ms | 10.12466ms | 228.74400ms | 74.52826ms | 161.53% | +2,358.30% |
| QueryBench | benchAsyncQueries | | [] | 1 | 10 | 4,439,184b | 4.63700ms | 9.31360ms | 6.30906ms | 20.17200ms | 4.89743ms | 52.58% | +396.22% |
| QueryBench | benchAsyncQueriesUsingPool | | [] | 1 | 10 | 6,633,522b | 3.53700ms | 4.33780ms | 3.87706ms | 6.75500ms | 0.98273ms | 22.65% | +131.12% |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+------------+------------+-------------+------------+---------+------------+
w/o zlib:
+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+
| benchmark | subject | groups | params | revs | its | mem_peak | best | mean | mode | worst | stdev | rstdev | diff |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+
| QueryBench | benchPdoQueries | | [] | 1 | 10 | 4,433,944b | 1.28300ms | 1.50140ms | 1.52774ms | 1.76400ms | 0.13752ms | 9.16% | 0.00% |
| QueryBench | benchSyncQueries | | [] | 1 | 10 | 4,433,952b | 4.29400ms | 7.52330ms | 4.86931ms | 15.60100ms | 4.11548ms | 54.70% | +401.09% |
| QueryBench | benchAsyncQueries | | [] | 1 | 10 | 4,433,952b | 3.94700ms | 4.96600ms | 4.40386ms | 9.93300ms | 1.68206ms | 33.87% | +230.76% |
| QueryBench | benchAsyncQueriesUsingPool | | [] | 1 | 10 | 6,592,336b | 2.97000ms | 3.20040ms | 3.27979ms | 3.35800ms | 0.12980ms | 4.06% | +113.16% |
+------------+----------------------------+--------+--------+------+-----+------------+-----------+-----------+-----------+------------+-----------+--------+----------+
Probably it must not be enabled by default when zlib is available.
Benchmark source: https://github.com/torinaki/mysql/blob/benchmarks/benchmarks/QueryBench.php
Hello!
I get a fatal error, when the Deadlock happens
I use Transaction class, with Connection class such problem is not present
I use 0.3.1 release
PHP Fatal error: Uncaught TypeError: Argument 1 passed to Amp\Mysql\DataTypes::decodeBinary() must be of the type integer, null given, called in /application/vendor/amphp/mysql/lib/Internal/Processor.php on line 979 and defined in /application/vendor/amphp/mysql/lib/DataTypes.php:80
Stack trace:
#0 /application/vendor/amphp/mysql/lib/Internal/Processor.php(979): Amp\Mysql\DataTypes::decodeBinary(NULL, '\x04#40001Deadlock...', NULL)
#1 /application/vendor/amphp/mysql/lib/Internal/Processor.php(1253): Amp\Mysql\Internal\Processor->handleBinaryResultsetRow('\xFF\xBD\x04#40001Deadlo...')
#2 /application/vendor/amphp/mysql/lib/Internal/Processor.php(240): Amp\Mysql\Internal\Processor->parsePayload('\xFF\xBD\x04#40001Deadlo...')
#3 /application/vendor/amphp/mysql/lib/Internal/Processor.php(216): Amp\Mysql\Internal\Processor->processData(Array)
#4 [internal function]: Amp\Mysql\Internal\Processor->read()
#5 /application/vendor/amphp/amp/lib/Coroutine.php(74): Generator->send('\x01\x00\x00\x01\x04C\x00\x00\x02\x03def\x11m...')
#6 /ap in /application/vendor/amphp/mysql/lib/DataTypes.php on line 80
I was not able yet to find the cause but I am able to reproduce it. In some situations queries never finish (neither failing nor succeeding).
What I found out is that the pool has only a single connection which is idle before the query is sent. So I guess the problem lies in the Processor, not in the pool.
Pool::init
might not return a promise, but null
instead. See failing test for amp_v2
branch.
See also http://chat.stackoverflow.com/transcript/message/36252000#36252000.
Hello
My table has 3504 rows.
When I use this code:
function resultToArray($result) {
$rows = array();
while($row = mysqli_fetch_assoc($result)) {
$rows[] = $row;
}
return $rows;
}
function GetMyConnectionGroup() {
$servername = 'localhost';
$username = 'root';
$password = '******';
$dbname = 'dbn';
global $g_link;
if ($g_link) return $g_link;
$g_link = new mysqli($servername, $username, $password, $dbname) or die('Could not connect to server.');
return $g_link;
}
$res = mysqli_query(GetMyConnectionGroup() , "SELECT * FROM groups ORDER BY id");
$rows = resultToArray($res);
mysqli_free_result($res);
I can receive all of 3504 rows
But when I use this code:
function MGSC($code='', $m=0) {
global $row;
global $db;
\Amp\Loop::run(function() use ($code, $row, $db, $m) {
global $row;
global $db;
if($code) {
if(!$db) $db = Amp\Mysql\pool("host=localhost user=root password=***** db=dbn");
$type = explode(' ', $code, 2);
$type[0] = strtolower($type[0]);
$result = yield $db->query($code);
if($type[0] == 'select') {
$row = [];
if($m === 0) {
while (yield $result->advance()) {
$row = $result->getCurrent();
}
}
elseif($m === 1) {
while (yield $result->advance()) {
$row[] = $result->getCurrent();
}
}
return $row;
}
else {
return $result;
}
}
else {
if($db) $db->close();
}
});
if($code) {if($row) return $row; elseif(isset($result)) return $result;}
}
$rows = MGSC("SELECT * FROM groups ORDER BY id", 1);
I mostly receive just 4 rows and some times 25 rows.
Also these errors:
2018-02-07T16:34:01.942293Z 215 [Note] Aborted connection 215 to db: 'dbn' user: 'root' host: 'localhost' (Got an error writing communication packets)
2018-02-07T15:48:02.011128Z 169 [Note] Aborted connection 169 to db: 'dbn' user: 'root' host: 'localhost' (Got an error reading communication packets)
How can I receive all rows ?
What is the problem ?
We updated the amphp/mysql from f077c40 to 419279e. Now we get the following exception from time to time.
I guess it could be related to connections closed by mysql server due to an idle timeout.
And it would be really cool if you tag your releases so one can use composer version deps.
Cannot reference an invalid watcher identifier: 'pk' (File: /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Loop/Driver.php:476) (Class: Amp\Loop\InvalidWatcherError)
[#1] Amp\Loop\Driver->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/lib/Sensphere/Amp/Loop/DecoratableDriver.php on line 141
[#2] Sensphere\Amp\Loop\DecoratableDriver->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Loop.php on line 288
[#3] Amp\Loop::reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/byte-stream/lib/ResourceInputStream.php on line 204
[#4] Amp\ByteStream\ResourceInputStream->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/socket/src/Socket.php on line 88
[#5] Amp\Socket\Socket->reference in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 184
[#6] Amp\Mysql\Internal\Processor->addDeferred in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 300
[#7] Amp\Mysql\Internal\Processor->Amp\Mysql\Internal\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 276
[#8] Amp\Mysql\Internal\Processor->appendTask in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 302
[#9] Amp\Mysql\Internal\Processor->startCommand in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Internal/Processor.php on line 337
[#10] Amp\Mysql\Internal\Processor->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Connection.php on line 98
[#11] Amp\Mysql\Connection->Amp\Mysql\{closure} in on line
[#12] Generator->current in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 39
[#13] Amp\Coroutine->__construct in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/functions.php on line 61
[#14] Amp\call in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Connection.php on line 109
[#15] Amp\Mysql\Connection->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Pool.php on line 268
[#16] Amp\Mysql\Pool->Amp\Mysql\{closure} in on line
[#17] Generator->current in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 39
[#18] Amp\Coroutine->__construct in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/functions.php on line 61
[#19] Amp\call in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/mysql/src/Pool.php on line 283
[#20] Amp\Mysql\Pool->query in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/lib/Sensphere/DB/MySql/AsyncPool.php on line 58
[#21] Sensphere\DB\MySql\AsyncPool->Sensphere\DB\MySql\{closure} in on line
[#22] Generator->throw in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 71
[#23] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#24] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#25] Amp\Coroutine->resolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 151
[#26] Amp\Coroutine->fail in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 93
[#27] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#28] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#29] Amp\Coroutine->resolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 151
[#30] Amp\Coroutine->fail in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Coroutine.php on line 93
[#31] Amp\Coroutine->Amp\{closure} in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Failure.php on line 26
[#32] Amp\Failure->onResolve in /home/ci-php2/builds/e0553b41/0/jaumo/jaumo/vendor/amphp/amp/lib/Internal/Placeholder.php on line 122
[#33] class@anonymous/home/ci-php
There should be a LICENSE
with the license in it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.