Comments (22)
Hello,
I have created a sample project which works for me (please grab it from here). Here is what I did:
- executed "ssh -L 9797:localhost:3306 my-username@mysql-server-hostname"
- "cd mysql-binlog-connector-java-using-scala" (sample project mentioned earlier)
- "./mvnw clean test" (you can use mvn instead of ./mvnw if you already have Maven)
Please try it yourself. You should get something similar to:
INFO: Connected to 127.0.0.1:9797 at mysql-bin.000002/107
connected
New event
New event
Let me know if you have any issues.
P.S. I've used different username and password (src/test/scala/junit.scala) so you might want to change them back before running the test.
from mysql-binlog-connector-java.
Hello,
Thank you for your help!
I tried your project and it doesn't work with my server, so I guess the problem is not with mysql-binlog-connector-java
.
I also don't get any of the lifecycle callbacks, the program just blocks after outputing this:
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running BinaryLogClientTest
So I guess there is some problem with the sql server. Do you have any idea on how I could debug this?
thanks!
from mysql-binlog-connector-java.
Hi again,
Could you please try mysql -h127.0.0.1 --port=9797 --protocol=TCP -uuser -ppass
? Is it working? Also, which OS do you use?
I think you don't get lifecycle callbacks called because BinaryLogClient gets stuck at the very beginning (while trying to connect to the socket). You may want to replace client.connect
with client.connect(3000)
(3s). At least this way it won't 'hang'.
from mysql-binlog-connector-java.
Hello,
Yes I can connect using mysql -h 127.0.0.1 --port 9797 --protocol=tCP --user user -p
What I can't do is connect directly using the mysqlbinlog
binary:
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
ERROR: Got error reading packet from server: Slave can not handle replication events with the checksum that master is configured to log; the first event 'mysql-bin.000701' at 4, the last event read from '/data/mysql/mysql-bin.000701' at 120, the last byte read from '/data/mysql/mysql-bin.000701' at 120.
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
I think this is because my mysql-client version is more recent than the servers version, although I am not sure this is the problem.
But I thought mysql-binlog-connector-java
used it's own Java library to connect to mysql, so this should not be a problem?
I am using ubuntu 13.10.
Thanks!
from mysql-binlog-connector-java.
Hi,
I should have asked this before, but what is the version of the MySQL server you trying to connect to?
from mysql-binlog-connector-java.
I am using 5.6.14.
from mysql-binlog-connector-java.
I need about 10 minutes to verify that tests pass against 5.6.14 release. In a meantime, do you think you could debug into BinaryLogClient.connect and check how far it goes (line number would be great)?
from mysql-binlog-connector-java.
Yep, tests passed. So this is not MySQL server version related.
from mysql-binlog-connector-java.
If you were willing to install vagrant you could:
git clone https://github.com/shyiko/mysql-binlog-connector-java.git
cd mysql-binlog-connector-java/
mvn clean verify \
-Dvagrant.integration.box=supplement/vagrant/mysql-5.6.12-sandbox-prepackaged
This way one can verify that mysql-binlog-connector-java works against sandboxed environment.
from mysql-binlog-connector-java.
Hi,
Thank you for your help!
I took a thread dump from ./mvnw clean test
and it looks like it's stuck getting the filename and position:
at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.read(ByteArrayInputStream.java:170)
at com.github.shyiko.mysql.binlog.io.ByteArrayInputStream.readInteger(ByteArrayInputStream.java:45)
at com.github.shyiko.mysql.binlog.network.protocol.PacketChannel.read(PacketChannel.java:55)
at com.github.shyiko.mysql.binlog.BinaryLogClient.readResultSet(BinaryLogClient.java:486)
at com.github.shyiko.mysql.binlog.BinaryLogClient.fetchBinlogFilenameAndPosition(BinaryLogClient.java:395)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:250)
Here's the full dump: https://gist.github.com/simao/8280188
I do have vagrant
installed and I tried to run that command but the vm provisioning fails because it doesn't seem to have internet access, it can't download the required packages. I'll try to look into this to make sure it's not something weird with my OS, but looks like the problem is with the mysql-server?
Thanks
from mysql-binlog-connector-java.
Well, whatever the cause is - mysql-binlog-connector-java shouldn't get stuck without an explanation. Also, thank you for the stack trace. BinaryLogClient passed greeting & authentication and got stuck on binlog filename/position resolution.
Could you please execute show master status
and tell me what the output is?
from mysql-binlog-connector-java.
here it is:
+------------------+-----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+-----------+--------------+------------------+-------------------+
| mysql-bin.000716 | 252731600 | | | |
+------------------+-----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
from mysql-binlog-connector-java.
Looks OK. I hoped that user just don't has the necessary permissions, but it seems it has. Let's take another way. I've just committed support for custom SocketFactory. Could you please try enabling/disabling Nagle algorithm (as shown below)?
client.setSocketFactory(new SocketFactory() {
override def createSocket = {
val socket = new Socket()
socket.setTcpNoDelay(true)
socket
}
})
from mysql-binlog-connector-java.
Actually, the user I use for mysqlbinlog
doesnt have permissions for show master status
! It has for enough permissions to read from the binlog but not for show master status
.
from mysql-binlog-connector-java.
I see. Could you please grant REPLICATION CLIENT permission to whatever user you use for BinaryLogClient ("SHOW MASTER STATUS" isn't necessary only when binlogFilename & binlogPosition are specified)?
from mysql-binlog-connector-java.
According to this you could also try
client.setBinlogFilename("")
client.setBinlogPosition(4)
but I haven't tried it myself (note that this is different from the "last binlog filename/position" resolution).
from mysql-binlog-connector-java.
Yep, setting the file name and position explicitly works fine. I was not aware of those options, I should have tried that :(
I will add the REPLICATION CLIENT
grant and chef provision the servers and try this with proper auth.
Thank you so much for your help.
from mysql-binlog-connector-java.
No problem. I'm sorry you had to go through all of this :)
from mysql-binlog-connector-java.
Hi!
I provisioned the servers and now the user has REPLICATION CLIENT and REPLICATION slave, but this still occurs if I don't explicitly set the filename and position. Do you know what the problem may be?
Thanks!
from mysql-binlog-connector-java.
Hello,
That's weird. I've created a new user (msandbox_slave) and granted him REPLICATION SLAVE and REPLICATION CLIENT privileges (as shown below). Integration tests passed (and failed once I revoke REPLICATION CLIENT, which is expected).
CREATE USER 'msandbox_slave' IDENTIFIED BY 'msandbox';
GRANT REPLICATION SLAVE ON *.* TO 'msandbox_slave';
GRANT REPLICATION CLIENT ON *.* TO 'msandbox_slave';
I would suggest to (in that order):
- make sure
mysql -hsame_host_as_for_binarylogclient -usame_user_as_for_binarylogclient -ppassword --port=9797 --protocol=TCP -e 'show master status'
returns data without any errors - try disabling Nagle algorithm as I wrote before
from mysql-binlog-connector-java.
OK! My bad, it works now! Permissions were the culprit then :)
Thanks again for your help!
from mysql-binlog-connector-java.
Hello,
date format not converting to elasticsearch date format, getting exception java.util.date cannot cast to string, unsafe domain object conversion.
Please help me out regarding this issue.
from mysql-binlog-connector-java.
Related Issues (20)
- NPE in TableMapEvent deserialization continuously HOT 1
- exception occured when i use function [JsonBinary.parseAsString]
- com.github.shyiko.mysql.binlog.network.AuthenticationException: Client does not support authentication protocol requested by server; consider upgrading MySQL client HOT 5
- ByteArrayInputStream has a performance problem HOT 2
- No break keyword in the switch-case code block.
- How to get only the binlog that listens to the specified db database HOT 1
- Is there any performance test report?
- BinaryLogClient auto detects wrongly MariaDB instead of MySQL HOT 1
- I want to get all the columns instead of filtering out the columns with a null value
- readMetadata HOT 1
- Provide ability to specify JDBC connection string HOT 1
- Failed to execute CommandLineRunner
- disconnect call getting blocked with keepalive thread
- [AWS Aurora] com.github.shyiko.mysql.binlog.network.AuthenticationException: unsupported authentication method: mysql_clear_password
- How to solve the binlog sync delay?
- The ROTATE event gets the wrong position
- Wrong value for time(6) data type
- Failed to deserialize data of EventHeaderV4
- [v0.25.3] Failed to deserialize data of EventHeaderV4. Possible fixes included.
- 字段类似是longtext,eventData.getRows()返回的是个地址
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mysql-binlog-connector-java.