Comments (6)
lizongbo
commented on June 15, 2024
升级到 1.2.19来试一下能否重现bug。最近的版本有连接相关的优化逻辑，以最新的包来测试验证为准吧。
from druid.
4fool
commented on June 15, 2024
升级到 1.2.19来试一下能否重现bug。最近的版本有连接相关的优化逻辑，以最新的包来测试验证为准吧。
收到。
DruidConnectionHolder.discard==false的连接，既有关闭的连接也有未关闭的连接，为了快速检测和分析是否有泄露的连接，建议在DruidConnectionHolder中包含几个字段：
- 用来表示是否已调用了close的字段
- 用来表示调用close是否出现异常的字段
- 用来表示调用close的时间戳字段
- 用来表示调用close的线程栈字段
有了以上信息，可以通过arthas vmtool一条命令查询出泄露的连接信息
from druid.
kimmking
commented on June 15, 2024
这是一个很久的bug，之前修复过多次了， @4fool 能否做一个可以复现的case？
from druid.
4fool
commented on June 15, 2024
这是一个很久的bug，之前修复过多次了， @4fool 能否做一个可以复现的case？
@kimmking 以下代码模拟了在设置了phyTimeoutMillis参数情况下，连接泄露的场景。
import com.alibaba.druid.pool.DruidConnectionHolder;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.pool.DruidPooledConnection;
import java.util.List;
import java.util.Map;
public class DruidAbandonedCase4PhyTimeout {
public static void main(String[] args) throws Exception{
DruidDataSource dataSource = new DruidDataSource();
dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
dataSource.setUsername("root");
dataSource.setPassword("123456");
dataSource.setUrl("jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false");
dataSource.setMinIdle(2);
// 保证不会触发DestroyTask，以便手动调用shrink
dataSource.setTimeBetweenEvictionRunsMillis(1 * 60 * 60 * 1000);
// 物理连接超时时长(超过该阈值会close掉物理连接)
dataSource.setPhyTimeoutMillis(1000);
dataSource.setMinEvictableIdleTimeMillis(500);
dataSource.setKeepAliveBetweenTimeMillis(501);
dataSource.init();
DruidPooledConnection connection1 = dataSource.getConnection();
DruidConnectionHolder holder1 = connection1.getConnectionHolder();
System.out.println(holder1);
try{
//假装该连接使用了500ms，要小于phyTimeoutMillis，以免归还连接的时候被discard了
Thread.sleep(500L);
}catch (Exception e){
e.printStackTrace();
}
DruidPooledConnection connection2 = dataSource.getConnection();
DruidConnectionHolder holder2 = connection2.getConnectionHolder();
System.out.println(holder2);
connection2.close();
connection1.close();
System.out.println("调用shrink【之前】，连接池中连接情况,begin");
List<Map<String, Object>> conns = dataSource.getPoolingConnectionInfo();
for(Map<String,Object> conn : conns){
System.out.println(conn.get("id"));
}
System.out.println("调用shrink【之前】，连接池中连接情况,end");
try{
// 让连接1的物理连接时间超过phyTimeoutMillis，以观察连接1是否会被连接池释放掉
Thread.sleep(501L);
}catch (Exception e){
e.printStackTrace();
}
System.out.println();
System.out.println("物理连接时间,begin");
long ctm = System.currentTimeMillis();
System.out.println(holder1 + " phyConnectTimeMillis:" + (ctm - holder1.getConnectTimeMillis()) + " ,idleMillis:" + (ctm - holder1.getLastActiveTimeMillis()));
System.out.println(holder2 + " phyConnectTimeMillis:" + (ctm - holder2.getConnectTimeMillis()) + " ,idleMillis:" + (ctm - holder2.getLastActiveTimeMillis()));
System.out.println("物理连接时间,end");
System.out.println();
// 手动执行shrink
dataSource.shrink(true);
System.out.println("调用shrink【之后】，连接池中连接情况,begin");
conns = dataSource.getPoolingConnectionInfo();
for(Map<String,Object> conn : conns){
System.out.println(conn.get("id"));
}
System.out.println("调用shrink【之后】，连接池中连接情况,end");
System.out.println();
System.out.println(holder1 + " isClosed:" + holder1.getConnection().isClosed());
System.out.println(holder2 + " isClosed:" + holder2.getConnection().isClosed());
}
}
运行以上程序，输出结果如下：
{ID:26004719, ConnectTime:"2023-10-10 12:40:04", UseCount:1, LastActiveTime:"2023-10-10 12:40:04"}
{ID:31314834, ConnectTime:"2023-10-10 12:40:05", UseCount:1, LastActiveTime:"2023-10-10 12:40:05"}
调用shrink【之前】，连接池中连接情况,begin
31314834
26004719
调用shrink【之前】，连接池中连接情况,end
物理连接时间,begin
{ID:26004719, ConnectTime:"2023-10-10 12:40:04", UseCount:1, LastActiveTime:"2023-10-10 12:40:05"} phyConnectTimeMillis:1011 ,idleMillis:501
{ID:31314834, ConnectTime:"2023-10-10 12:40:05", UseCount:1, LastActiveTime:"2023-10-10 12:40:05"} phyConnectTimeMillis:502 ,idleMillis:501
物理连接时间,end
调用shrink【之后】，连接池中连接情况,begin
26004719
调用shrink【之后】，连接池中连接情况,end
{ID:26004719, ConnectTime:"2023-10-10 12:40:04", UseCount:1, LastActiveTime:"2023-10-10 12:40:05"} isClosed:true
{ID:31314834, ConnectTime:"2023-10-10 12:40:05", UseCount:1, LastActiveTime:"2023-10-10 12:40:05"} isClosed:false
期望执行结果：
ID是26004719的连接应该从连接池中删除，ID是31314834的连接应该被连接池管理
实际执行结果：
ID是26004719的连接没有从连接池删除；ID是31314834的连接却从连接池删除了，且连接没有被关闭，即ID是31314834的连接既没有被连接池管理又没有被关闭，属于泄露的连接。
测试了druid 1.1.11，1.2.0，1.2.8 - 1.2.17，1.2.18-1.2.20，其中1.1.11，1.2.0，1.2.8 - 1.2.17都存在以上连接泄露的问题，1.2.18-1.2.20不存在以上连接泄露的问题了。
from druid.
4fool
commented on June 15, 2024
以下代码模拟当数据库操作出现FatalError的情况下出现连接泄露的场景。
import com.alibaba.druid.pool.DruidConnectionHolder;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.pool.DruidPooledConnection;
import com.alibaba.druid.pool.ExceptionSorter;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.List;
import java.util.Map;
import java.util.Properties;
public class DruidAbandonedCase4OnFatalError {
public static void main(String[] args) throws Exception{
DruidDataSource dataSource = new DruidDataSource();
dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
dataSource.setUsername("root");
dataSource.setPassword("123456");
dataSource.setUrl("jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false");
dataSource.setMinIdle(2);
// 保证不会触发DestroyTask，以便手动调用shrink
dataSource.setTimeBetweenEvictionRunsMillis(1 * 60 * 60 * 1000);
dataSource.setMinEvictableIdleTimeMillis(0);
// 用来mock出现FatalError的数据库异常
dataSource.setExceptionSorter(new MockExceptionSorter());
dataSource.init();
// connectioin1为正常的连接
DruidPooledConnection connection1 = dataSource.getConnection();
DruidConnectionHolder holder1 = connection1.getConnectionHolder();
System.out.println(holder1.getConnectionId());
// connectioin2是数据库操作出现异常的连接
DruidPooledConnection connection2 = dataSource.getConnection();
DruidConnectionHolder holder2 = connection2.getConnectionHolder();
System.out.println(holder2.getConnectionId());
// 数据库操作出现FatalError
fatalError(connection2);
connection2.close();
// connectioin3是正常的连接
DruidPooledConnection connection3 = dataSource.getConnection();
DruidConnectionHolder holder3 = connection3.getConnectionHolder();
System.out.println(holder3.getConnectionId());
connection3.close();
connection1.close();
System.out.println();
System.out.println("调用shrink【之前】，连接池中连接情况,begin");
List<Map<String, Object>> conns = dataSource.getPoolingConnectionInfo();
for(Map<String,Object> conn : conns){
System.out.println(conn.get("connectionId"));
}
System.out.println("调用shrink【之前】，连接池中连接情况,end");
// 手动执行shrink
dataSource.shrink(true);
System.out.println();
System.out.println("调用shrink【之后】，连接池中连接情况,begin");
conns = dataSource.getPoolingConnectionInfo();
for(Map<String,Object> conn : conns){
System.out.println(conn.get("connectionId"));
}
System.out.println("调用shrink【之后】，连接池中连接情况,end");
System.out.println();
System.out.println(holder1.getConnectionId() + " isClosed:" + holder1.getConnection().isClosed());
System.out.println(holder2.getConnectionId() + " isClosed:" + holder2.getConnection().isClosed());
System.out.println(holder3.getConnectionId() + " isClosed:" + holder3.getConnection().isClosed());
}
private static void fatalError(Connection conn){
Statement pstmt = null;
ResultSet rs = null;
try {
String sql = "select * from unExistTable";
pstmt = conn.createStatement();
rs = pstmt.executeQuery(sql);
while (rs.next()) {
int id = rs.getInt("id");
System.out.println("id: " + id);
}
} catch (SQLException e) {
// e.printStackTrace();
} finally {
try {
if (rs != null) {
rs.close();
}
if (pstmt != null) {
pstmt.close();
}
} catch (SQLException e) {
// e.printStackTrace();
}
}
}
static class MockExceptionSorter implements ExceptionSorter {
@Override
public boolean isExceptionFatal(SQLException e) {
return true;
}
@Override
public void configFromProperties(Properties properties) {
}
}
}
运行以上程序，输出结果如下：
10001
10002
严重: {conn-10002} discard
java.sql.SQLSyntaxErrorException: Table 'test.unexisttable' doesn't exist
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:121)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1200)
at com.alibaba.druid.pool.DruidPooledStatement.executeQuery(DruidPooledStatement.java:300)
at DruidAbandonedCase4OnFatalError.fatalError(DruidAbandonedCase4OnFatalError.java:82)
at DruidAbandonedCase4OnFatalError.main(DruidAbandonedCase4OnFatalError.java:40)
10003
调用shrink【之前】，连接池中连接情况,begin
10003
10001
调用shrink【之前】，连接池中连接情况,end
调用shrink【之后】，连接池中连接情况,begin
10001
调用shrink【之后】，连接池中连接情况,end
10001 isClosed:true
10002 isClosed:true
10003 isClosed:false
代码执行逻辑描述：
- 获取connectionId为1001的连接；
- 获取connectionId为1002的连接，1002在执行数据库操作的时候出现了FatalError，然后调用close方法归还连接；
- 获取connectionId为1003的连接，调用close归还连接；
- 调用close归还connectionId为1001的连接；
- 打印出连接池管理的连接有哪些；
- 调用数据库连接池的shrink方法；
- 打印出连接池管理的连接有哪些；
- 打印出每个连接是否已被关闭。
期望执行结果：
- 由于connectionId为1002的连接发生了FatalError，该连接会被关闭且从连接池中删除；
- 连接池管理的连接包括connectionId为1001和1003，且连接状态都是非关闭状态
实际执行结果：
- connectionId为1002的连接已被关闭且从连接池删除【符合期望】；
- 连接池管理的连接只包括connectionId为1001的连接，不包含connectionId为1003的连接（泄露的连接）【不符合期望】；
- connectionId为1001的连接为已关闭状态（不应该关闭掉）【不符合期望】;
测试了druid 1.1.11，1.2.8-1.2.17，1.2.18-1.2.20，其中1.2.8-1.2.17都存在以上连接泄露的问题，1.1.11，1.2.18-1.2.20不存在以上连接泄露的问题。
from druid.
4fool
commented on June 15, 2024
以下代码模拟当配置了keepalive选项的情况下出现连接泄露的场景。
import com.alibaba.druid.pool.DruidConnectionHolder;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.pool.DruidPooledConnection;
import java.lang.reflect.Field;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DruidAbandonedCase4Keepalive {
public static void main(String[] args) throws Exception {
Map<Long,DruidConnectionHolder> holderMap = new HashMap<>();
DruidDataSource dataSource = new DruidDataSource();
dataSource.setDriverClassName("com.mysql.cj.jdbc.Driver");
dataSource.setUsername("root");
dataSource.setPassword("123456");
dataSource.setUrl("jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false");
dataSource.setMinIdle(2);
dataSource.setKeepAlive(true);
dataSource.setTimeBetweenEvictionRunsMillis(500);
long minEvictableIdleTimeMillis = 500L;
dataSource.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
long keepAliveBetweenTimeMillis = 1000L;
dataSource.setKeepAliveBetweenTimeMillis(keepAliveBetweenTimeMillis);
dataSource.init();
try{
Field destroyConnectionThreadField = DruidDataSource.class.getDeclaredField("destroyConnectionThread");
destroyConnectionThreadField.setAccessible(true);
Thread destroyConnectionThread = (Thread)destroyConnectionThreadField.get(dataSource);
destroyConnectionThread.interrupt();
Thread.State state = destroyConnectionThread.getState();
System.out.println("destroyConnectionThread state : " + state);
}catch (Exception e){
e.printStackTrace();
}
DruidPooledConnection connection1 = dataSource.getConnection();
DruidConnectionHolder holder1 = connection1.getConnectionHolder();
print(holder1);
holderMap.put(holder1.getConnectionId(),holder1);
DruidPooledConnection connection2 = dataSource.getConnection();
DruidConnectionHolder holder2 = connection2.getConnectionHolder();
print(holder2);
holderMap.put(holder2.getConnectionId(),holder2);
connection2.close();
sleep(keepAliveBetweenTimeMillis - minEvictableIdleTimeMillis + 100L);
connection1.close();
sleep(minEvictableIdleTimeMillis - 100L);
pooling(dataSource,holderMap);
shrink(dataSource);
pooling(dataSource,holderMap);
sleep(100L);
shrink(dataSource);
pooling(dataSource,holderMap);
System.out.println();
print(holder1);
print(holder2);
}
private static void sleep(long millis){
try{
Thread.sleep(millis);
}catch (InterruptedException e){
e.printStackTrace();
}
}
private static void shrink(DruidDataSource dataSource){
System.out.println();
System.out.println("调用shrink");
// 手动执行shrink
dataSource.shrink(true);
}
private static void pooling(DruidDataSource dataSource,Map<Long,DruidConnectionHolder> holderMap) throws SQLException {
System.out.println();
System.out.println("连接池中连接情况,begin");
List<Map<String, Object>> conns = dataSource.getPoolingConnectionInfo();
for (Map<String, Object> conn : conns) {
print(holderMap.get(conn.get("connectionId")));
}
System.out.println("连接池中连接情况,end");
}
private static void print(DruidConnectionHolder holder) throws SQLException {
System.out.println(holder.getConnectionId() + /*" : " + holder +*/
" idleMillis : " + (System.currentTimeMillis() - holder.getLastActiveTimeMillis()) + " isClosed:" + holder.getConnection().isClosed());
}
}运行以上程序，输出结果如下(druid 1.2.8)：
destroyConnectionThread state : TIMED_WAITING
10001 idleMillis : 6 isClosed:false
10002 idleMillis : 1 isClosed:false
连接池中连接情况,begin
10002 idleMillis : 1015 isClosed:false
10001 idleMillis : 404 isClosed:false
连接池中连接情况,end
调用shrink
连接池中连接情况,begin
10001 idleMillis : 405 isClosed:false
10002 idleMillis : 1016 isClosed:false
连接池中连接情况,end
调用shrink
连接池中连接情况,begin
10001 idleMillis : 405 isClosed:false
10002 idleMillis : 1016 isClosed:false
连接池中连接情况,end
调用shrink
连接池中连接情况,begin
10002 idleMillis : 1133 isClosed:true
连接池中连接情况,end
10001 idleMillis : 522 isClosed:false
10002 idleMillis : 1133 isClosed:true代码执行逻辑描述：
- 为了便于测试，中断Druid-ConnectionPool-Destroy-xx线程，以便手动调用shrink；
- 获取connectionId为1001的连接，打印连接信息；
- 获取connectionId为1002的连接，打印连接信息；
- 调用close，归还connectionId为1002的连接
- sleep (keepAliveBetweenTimeMillis - minEvictableIdleTimeMillis + 100L)时长
- 调用close，归还connectionId为1001的连接
- sleep (minEvictableIdleTimeMillis - 100L)时长（两个sleep是为了让1002 idleMillis大于keepAliveBetweenTimeMillis，让1001小于minEvictableIdleTimeMillis）
- 打印连接池中连接信息
- 执行shrink
- 打印连接池中连接信息
- sleep 100ms
- 执行shrink
- 打印连接池中连接信息
- 打印1001和1002连接信息
期望执行结果：
- connectionId为1001和1002的连接都应该被连接池管理，且连接状态都应该是未关闭状态
实际执行结果：
- connectionId为1001的连接已从连接池中删除，且连接状态是未关闭状态，泄露的连接【不符合期望】；
- connectionId为1002的连接虽然在连接池中，但是连接状态是关闭状态【不符合期望】。
测试了几个版本，druid 1.1.16-1.1.24，1.2.0-1.2.17都存在连接泄露问题，druid 1.2.18-1.2.20不存在连接泄露问题。
from druid.
Related Issues (20)
- dbtype null HOT 1
- druid连接达梦8数据库，执行merge into语法报错 HOT 1
- Druid 密码加密后，nacos热部署刷新，导致数据库连接失败 报Access Denied HOT 3
- DruidDataSource.configFromPropeties缺失配置项 HOT 1
- initedLatch一次性锁支持不了restart重启后再次init初始化 HOT 1
- Parse PL/SQL Parameter Declaration: NOT NULL Failure
- PL/SQL SQLSetStatement "a := 1" 输出是 "SET a to 1"
- Oracle PL/SQL while loop statement语法问题
- DruidDataSource 锁释放问题 HOT 1
- 连接池数据结构采用线程对象做key的map，实测能够有效提升连接池处理性能 HOT 8
- 使用TOMCAT的JNDI的方式，无法设置socketTimeout和connectionTimeout两个参数 HOT 1
- 使用spring-native构建本地镜像运行报错 HOT 6
- druid parser support clickhouse dialect HOT 1
- 报错：maxEvictableIdleTimeMillis must be grater than minEvictableIdleTimeMillis HOT 5
- Cannot parse Oracle PL/SQL explicit cursor RETURN statement
- 无法解析Oracle PL/SQL Reverse For Loop
- WallFilter添加NoOpWallProvider实现或开放是否支持接口 HOT 4
- shrink方法中某个条件前后不一致 HOT 2
- 在Insert语句中columns添加注释不生效 HOT 1
- spring3.0.2以上版本+nacos +druid，配置nacos动态刷新后刷新任意一个无关的key都会导致druid连接断开并无法重连。 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from druid.