Comments (11)
even if the commits takes hundreds of milliseconds?
pgx does send the cancel signal. But again, it is very timing dependent and I don't know how likely successfully cancelling a commit
is at the PostgreSQL level.
You could probably experiment with a couple psql
sessions and pg_cancel_backend
to get more of a feel for how cancellation works.
But ultimately, it's not possible to know for certain what happened on the server without waiting to receive the response from the server. And context cancellation immediately cancels the query without waiting for a response.
Your options are:
- Remove the deadline altogether
- Customize the context cancellation behavior such that it requests the cancellation, but waits a while before severing the connection.
- Query the database after the interrupted commit to see if the change was applied.
from pgx.
I've read the code carefully and now that I understand more or less what and how it does it, you're right, theres no way to actually guarantee a cancellation in case of context deadline/cancel. for now, making sure the data is consistent is more important to me than performance so I'll remind myself to use context.Background() for commit/rollback operations.
Thanks for your quick replies. pgx is awesome!
from pgx.
You set a timeout of 1ms. That is long enough to send the COMMIT
command to PostgreSQL, but apparently not long a receive the result back from PostgreSQL. So your command is interrupted on the Go side, but PostgreSQL still received the COMMIT
.
from pgx.
that is just reproducer. in production we can see commit taking a few milliseconds sometimes (50, 100 or more, under load). From the clients POV, it looks like we cancel the context but the tx goes through
from pgx.
Regardless of the exact timing, that is still almost certainly what is happening. The COMMIT
is sent, but the command is canceled before the response is received.
from pgx.
Do you suggest not using a context with cancellation/deadline for tx.Commit
? I understand that this might not be a bug, but it can cause problems...
from pgx.
on pgx's side, what happens while commit is running and it receives a cancellation via context? how does it abort an in-progress commit?
from pgx.
on pgx's side, what happens while commit is running and it receives a cancellation via context? how does it abort an in-progress commit?
It sets a deadline on the net.Conn
. This will immediately cause any in-progress network read or writes to fail. pgx will respond the the error by closing the net.Conn
.
But again, if the COMMIT
has already been transmitted over the network there is no way to stop it. (Technically, there is an out of band query cancellation system, and pgx does use it, but the chances of that stopping a COMMIT
are almost zero.)
from pgx.
You can customize the cancellation behavior with pgconn.BuildContextWatcherHandler
, DeadlineContextWatcherHandler
and CancelRequestContextWatcherHandler
. But there is a fundamental issue of once the message is sent it can't be recalled.
You either wait for the response from the server or you cancel without knowledge of whether the command reached the server or not.
from pgx.
even if the commits takes hundreds of milliseconds? anyway, I'll remove the deadline from commit for the time being. This was a fun bug I had to find the hard way
from pgx.
Should this example commit, then? this still commits the changes even though I think that it should not?
tx, err := pool.Begin(context.Background())
if err != nil {
slogFatal("failed to start tx", "error", err)
}
_, err = tx.Exec(context.Background(), "INSERT INTO mytable (id) VALUES (1);")
if err != nil {
slogFatal("failed to exec INSERT", "error", err)
}
ctx, cancel := context.WithTimeout(context.Background(), time.Millisecond*100)
defer cancel()
_, err = tx.Exec(ctx, "select pg_sleep(1);commit")
//err = tx.Commit(ctx)
from pgx.
Related Issues (20)
- pgxpool: different MaxConnLifetime for fallback hosts HOT 1
- The type for "MONEY" isn't detected
- Support directly inserting pgtype.Bits into a column of type BIT(n) using named args
- Discussion: useful to return rows early using iter.Seq? HOT 2
- Discussion: Improve pgx.CollectRows to dynamically allocate sufficient slice capacity to avoid memory reallocations HOT 2
- pgtype.ArrayCodec should support non-comma delimiter HOT 2
- `fatal error: concurrent map writes` in `EnumCodec.lookupAndCacheString` HOT 4
- Support for require_auth configuration parameter HOT 1
- Security: Uncaught Exception Violation found by Snyk HOT 3
- RowToStructByName, Lax and Pos not working with `db:"-"` tag HOT 1
- Examples or documentation for "WHERE field IN $1" - should be "WHERE field IN ($1)" HOT 1
- Tracelog truncation can make output longer.
- dangling transactions after 'failed to deallocate cached statement' errors HOT 1
- Clarify ErrIsClosed return type in docs HOT 1
- v4.18.3 Security vulnerability CVE-2024-27289 HOT 2
- Running lib/pq and pgx in the same code base? HOT 1
- Cannot scan/insert user defined UUIDs.
- Memory Leak doing INSERT on High-Load service HOT 1
- Constructor function for pgtype.Interval or additional comments HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pgx.