Comments (23)
Hi @YoungHu
I saw your pull request and wanted to thank you for contributing!
Before we get it approve and merge in, I'd just like to do some more testing on our side.
Thank you!
from aws-mysql-jdbc.
I have also encountered this situation. aws-mysql-jdbc 1.1.8
from aws-mysql-jdbc.
Hi,
We also observed the issue in all the versions after 1.4.Please have a look on the attached screenshot.our observation were a simple insert will not reproduce the issue in our laod env. but when there are select and insert we are able to reproduce.
from aws-mysql-jdbc.
Hi @YoungHu,
Thanks for reaching out and raising this issue.
We'll take a look at this and keep you updated as we investigate.
Thank you for your patience!
from aws-mysql-jdbc.
Hi @YoungHu
I took a closer look at the Monitor
class and the uses of Queue<MonitorConnectionContext>
. While the stopMonitoring()
method does not call remove()
to remove the context from the queues, the contexts are removed from the queues in the run()
method by calling poll()
on the queues.
https://github.com/awslabs/aws-mysql-jdbc/blob/main/src/main/user-impl/java/com/mysql/cj/jdbc/ha/plugins/Monitor.java#L150
https://github.com/awslabs/aws-mysql-jdbc/blob/main/src/main/user-impl/java/com/mysql/cj/jdbc/ha/plugins/Monitor.java#L184
I also tried reproduce a memory leak, but so far unable to.
I made a simple Spring Boot application using aws-mysql-jdbc:1.1.7
and profiled it using JProfiler.
Below is a graph of the number of instances of the MonitorConnectionContext
class over time.
In the graph, there is a continuous pattern of the number of instances rising, then falling, then rising again. Not captured in the screenshot, but I did not observe any trends of the peaks getting higher over time.
I was hoping you could you provide the following details?
- Is your application using any framework, libraries, and/or connection pools? What are they?
- What type of queries is your application making? Are the long running? Transactional?
- Is your application using any non-default garbage collection settings/options?
- Anything else you think may be useful
Thank you for patience!
from aws-mysql-jdbc.
Hi @YoungHu
I took a closer look at the
Monitor
class and the uses ofQueue<MonitorConnectionContext>
. While thestopMonitoring()
method does not callremove()
to remove the context from the queues, the contexts are removed from the queues in therun()
method by callingpoll()
on the queues. https://github.com/awslabs/aws-mysql-jdbc/blob/main/src/main/user-impl/java/com/mysql/cj/jdbc/ha/plugins/Monitor.java#L150 https://github.com/awslabs/aws-mysql-jdbc/blob/main/src/main/user-impl/java/com/mysql/cj/jdbc/ha/plugins/Monitor.java#L184I also tried reproduce a memory leak, but so far unable to. I made a simple Spring Boot application using
aws-mysql-jdbc:1.1.7
and profiled it using JProfiler. Below is a graph of the number of instances of theMonitorConnectionContext
class over time.In the graph, there is a continuous pattern of the number of instances rising, then falling, then rising again. Not captured in the screenshot, but I did not observe any trends of the peaks getting higher over time.
I was hoping you could you provide the following details?
- Is your application using any framework, libraries, and/or connection pools? What are they?
- What type of queries is your application making? Are the long running? Transactional?
- Is your application using any non-default garbage collection settings/options?
- Anything else you think may be useful
Thank you for patience!
Hi @aaronchung-bitquill
we have 8 applications use 1.1.7 version, just one application have this problem.
following details as blew:
- SpringBoot 2.x, Druid 1.2.9, Mybatis 3.5.4.
- just simple sql select by key
- GC setting like this:-XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+DisableExplicitGC -XX:+CMSParallelRemarkEnabled
normally, run
method will call poll()
to remove item in queue, but when debug the code, I find run
method will break out sometimes, I have not idea what cause it. after break out, queue will just add not remove.
from aws-mysql-jdbc.
Hi @YoungHu
A few additional questions for the application that is experiencing this issue
- Can you give me some details about the database? Is it MySQL or Aurora MySQL? What versions?
- How is the DB URL specified?
- What plugins are being used?
- What database and connection related configurations do you have?
- Are there any other details about this particular application that is different than the others are working without issue?
Also, would u be able to enable driver logging and provide some driver logs?
Thank you
from aws-mysql-jdbc.
Hi @YoungHu
A few additional questions for the application that is experiencing this issue
- Can you give me some details about the database? Is it MySQL or Aurora MySQL? What versions?
- How is the DB URL specified?
- What plugins are being used?
- What database and connection related configurations do you have?
- Are there any other details about this particular application that is different than the others are working without issue?
Also, would u be able to enable driver logging and provide some driver logs?
Thank you
we are using Aurora MySQL 5.7 version,DB url is cluster endpoint
from aws-mysql-jdbc.
Hi @YoungHu
I suspect that it might be InterruptedException that is breaking the while loop in the Monitor#run
method. But unfortunately, I am still unable to reproduce the issue.
If possible would you be able to provide driver logs of when the issue occurs? That would greatly help us be able to confirm the source of the issue. Otherwise, would you be willing to share a sample app that reproduces the issue?
Thank you
from aws-mysql-jdbc.
Hi @YoungHu
I suspect that it might be InterruptedException that is breaking the while loop in the
Monitor#run
method. But unfortunately, I am still unable to reproduce the issue.If possible would you be able to provide driver logs of when the issue occurs? That would greatly help us be able to confirm the source of the issue. Otherwise, would you be willing to share a sample app that reproduces the issue?
Thank you
how to set logger? just like jdbc:mysql:aws://db-cluster-id:3306?key1=value&key2=value2& logger=com.mysql.cj.log.StandardLogger
?
from aws-mysql-jdbc.
HI @YoungHu
Apologies for the late reply.
Yes, that is correct.
from aws-mysql-jdbc.
Hi @YoungHu
I wanted to check in to see how things are going? Were you able to obtain driver logs?
from aws-mysql-jdbc.
Hi @YoungHu
We have not heard back from you for some time and wanted to check back in.
If there are no further updates on this ticket in the next 3 days, we will close this ticket. However, if you require further assistance, you can reopen this issue, or create a new one.
Thank you!
from aws-mysql-jdbc.
Hi @YoungHu
We will be closing this ticket now due to lack of updates. However, if you require further assistance, you can reopen this issue, or create a new one.
Thank you!
from aws-mysql-jdbc.
Hello,
I have the same problem upgrading from 1.1.0 to 1.1.11.
Please take a look to the entity of the memory leak:
Thank you,
Fabrizio
from aws-mysql-jdbc.
Hi @ftasso,
Thank you for letting us know you've experienced this. Your screenshot indicates there are many objects in memory, but that may not necessarily mean those objects were leaked. Leaked objects are usually shown as a long uptrend line of consumed memory, would you be able to verify whether this was the case?
Thank you!
from aws-mysql-jdbc.
Hello,
I made some tests about this issue and I confirm you that the instances of MonitorConnectionContext are not released.
I made a simply thread that make a full GC cleaner every 5 minutes but the allocated objects still remain allocated.
Details about my environment:
Thank you,
Fabrizio
from aws-mysql-jdbc.
Hi @ftasso,
Thanks for confirming that, we'll take another look at this and keep you updated as we investigate.
Thank you for your patience!
from aws-mysql-jdbc.
So, are you currently addressing this issue? What good temporary solutions do I have now?
from aws-mysql-jdbc.
Hi @KerouacEZ,
We have received similar reports on this issue for the AWS JDBC Driver. While we are still working on the root cause of the issue for both drivers, we have released a new version (v2.3.1) for the the AWS JDBC Driver that alleviates the issues you are seeing.
Once we have a fix for the issue, we will make it available for both the AWS JDBC Driver and the AWS JDBC Driver for MySQL. In the meantime, we suggest that you try out the AWS JDBC Driver with the MySQL Connector/J as the underlying driver. There is a migration guide on moving from the AWS JDBC Driver for MySQL to the AWS JDBC Driver available here.
Please let us know if you run into any integration issues.
from aws-mysql-jdbc.
Hi all,
We've recently added an experimental Host Monitoring Plugin (PR #764) to the AWS JDBC Driver, called the Host Monitoring Plugin v2. More information can be found here. The Host Monitoring Plugin v2 is functionally equivalent to the Host Monitoring Plugin and they are configured using the same parameters. The Host Monitoring Plugin v2 was created to address the issues that you all have been experiencing, and it is meant to be a more stable version of the Host Monitoring Plugin.
As it is currently an experimental plugin, please note that it should be tested before being used in a production environment. This plugin is not currently available on the AWS JDBC Driver for MySQL, but it can be tried out by using the snapshot build of the AWS JDBC Driver: aws-advanced-jdbc-wrapper-2.3.2-20231213.213138-10.jar
. For more information on using a snapshot build, see this page. If you are able to test out the plugin, please let us know if you encounter any errors or if this plugin was able to resolve the issues you've encountered.
Thank you!
from aws-mysql-jdbc.
Hi all, the aforementioned experimental Host Monitoring Plugin v2 is now available on our latest release of the driver. It is currently an experimental plugin so please test it out before using it in a production environment.
We will be closing this issue now, please don't hesitate to comment or open a new ticket if this issue persists with the experimental plugin or if you encounter any other issues.
from aws-mysql-jdbc.
We've observed the same memory leak issue with aws-mysql-jdbc version 1.1.10. After upgrade to 1.1.12, we might not reproduce the memory leak anymore. Even when default settings are met or explicitly defined as:
- connectionPluginFactories = true
- connectionPluginFactories = software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.failover.FailoverConnectionPluginFactory, software.aws.rds.jdbc.mysql.shading.com.mysql.cj.jdbc.ha.plugins.NodeMonitoringConnectionPluginFactory
(Documented here https://github.com/awslabs/aws-mysql-jdbc/blob/main/README.md#connection-plugin-manager-parameters)
-> It seems to be resolved in 1.1.12 with default settings.
from aws-mysql-jdbc.
Related Issues (20)
- Random IndexOutOfBoundsException occurences HOT 5
- Redundant "Reset stmt" when setting useServerPrepStmts&cachePrepStmts to true HOT 2
- Failover with a autocommit=false setup lead to broken connections HOT 8
- Setting allowMultiQueries leads to different results HOT 4
- BatchUpdateException.getUpdateCounts returns unexpected value -1 HOT 6
- Be more verbose about connecting to the read-only endpoint of an aurora RDS instance. HOT 2
- Default clusterId doesn't include user HOT 2
- During batchUpdate() Single line comments are causing error "Statement.executeUpdate() or Statement.executeLargeUpdate() cannot issue statements that produce result sets." HOT 4
- Potentially corrupt connection in pool HOT 4
- getHoldability() and getResultSetHoldability() return different results HOT 2
- Statement.getGeneratedKeys() returns unexpected value HOT 3
- Error Initializing DB Connection to Aurora DB in `AWS-CN` without `enableClusterAwareFailover=false` HOT 11
- Setting a large timeout leads to errors when executing SQL HOT 1
- Execution of executeUpdate Closes Previously Acquired ResultSet HOT 2
- Statement.getResultSetType () seems failed to change the result set type HOT 1
- Statement.cancel() does not work HOT 3
- what is the different between aws-mysql-jdbc and aws-advanced-jdbc-wrapper HOT 4
- docs: clarify differences between MySQL JDBC and JDBC Wrapper
- Is it possible to exclude Readers with different instance types if they are included? HOT 2
- NodeMonitoringConnectionPlugin causes Exceptions HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-mysql-jdbc.