Comments (40)
Yeah, I filed https://bugs.ruby-lang.org/issues/7215 with Ruby about this but I don't think it's gonna get fixed there. @petergeoghegan has a patch to Postgres with some workarounds for 9.6, and from what I can tell it is getting back-patched to older releases, too. Of course, this doesn't help with RDS if that is indeed the issue here.
from crystal-pg.
Can you share \d my_table from psql? If you don't want to share pubiclally
my email is will at bitfission .com
On Wednesday, May 11, 2016, Serdar Dogruyol [email protected]
wrote:
I'm trying to connect to my RDS instance with credentials like below.
DB = PG.connect("postgres://user_name:[email protected]/db_name?sslmode=require")DB.exec("SELECT * from my_table")
It just segfaults without any stacktrace.
Program exited because of a segmentation fault (11)
I'm clueless and need some help :/
β
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/will/crystal-pg/issues/40
from crystal-pg.
@will emailed!
from crystal-pg.
Got it thanks.
- if this is on --release, can you try without to get more information
- is it just that
select * from
query, or does evenselect now()
segfault? - are you using the last released version or head or a different version?
and finally if there isn't anything obvious
module PQ
class Connection
def read(frame_type)
size = read_i32
slice = read_bytes(size - 4)
frame = Frame.new(frame_type.not_nil!, slice).tap { |f| p f }
handle_async_frames(frame) ? read : frame
end
end
end
that will monkey patch in printing every frame from the wire, which could help track where the failure is. That you might also want to put in a secret gist
from crystal-pg.
Hey there,
- This wasnt on
--release
- Even
select now()
segfaults - I've tried both latest and master. Currently trying with master.
Even with monkey patching with above i can't see any stacktrace π’
from crystal-pg.
That makes me think it's dying in ssl negotiation. Try this?
module PG
class Connection
private def negotiate_ssl
puts "in negotiate" # new
write_i32 8
write_i32 80877103
@soc.flush
puts "sent ssl request" # new
c = @soc.read_char
puts "response #{c.inspect}" # new
serv_ssl = case c
when 'S' then true
when 'N' then false
else
raise ConnectionError.new(
"Unexpected SSL response from server: #{c.inspect}")
end
pp serv_ssl #new
if serv_ssl
@soc = OpenSSL::SSL::Socket.new(@soc, sync_close: true)
end
if @conninfo.sslmode == :require && !@soc.is_a?(OpenSSL::SSL::Socket)
close
raise ConnectionError.new("sslmode=require and server did not establish SSL")
end
end
end
end
from crystal-pg.
Unfortunately even this gives no output, only segfault :/
from crystal-pg.
You could get a backtrace for the segfault. At least on mac osx it's show in the Console app. I believe you can do the same in linux with gdb.
from crystal-pg.
@asterite how can i get that backtrace on OS X?
from crystal-pg.
Oh, actually... since we now have a sigfault handler that should appear when the program crashes, though I'm not sure it's as complete as the one Console used to show...
from crystal-pg.
@asterite Well i'm just getting this Program exited because of a segmentation fault (11)
.
Is there any way to find the root?
from crystal-pg.
There's lldb
/tmp β€ cat seg.cr
a = Pointer(UInt8).new(0)
a[0] = 3_u8
/tmp β€ crystal build seg.cr
/tmp β€ lldb seg
(lldb) target create "seg"
Current executable set to 'seg' (x86_64).
(lldb) r
Process 18090 launched: '/private/tmp/seg' (x86_64)
Process 18090 stopped
* thread #1: tid = 0x53aa38f, 0x0000000100007c35 seg`*Pointer(UInt8)@Pointer(T)#[]=<Pointer(UInt8), Int32, UInt8>:UInt8 + 21, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0000000100007c35 seg`*Pointer(UInt8)@Pointer(T)#[]=<Pointer(UInt8), Int32, UInt8>:UInt8 + 21
seg`*Pointer(UInt8)@Pointer(T)#[]=<Pointer(UInt8), Int32, UInt8>:UInt8:
-> 0x100007c35 <+21>: movb %cl, (%rax)
0x100007c37 <+23>: movb %cl, %al
0x100007c39 <+25>: addq $0x10, %rsp
0x100007c3d <+29>: popq %rbp
(lldb) bt
* thread #1: tid = 0x53aa38f, 0x0000000100007c35 seg`*Pointer(UInt8)@Pointer(T)#[]=<Pointer(UInt8), Int32, UInt8>:UInt8 + 21, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000100007c35 seg`*Pointer(UInt8)@Pointer(T)#[]=<Pointer(UInt8), Int32, UInt8>:UInt8 + 21
frame #1: 0x0000000100001455 seg`__crystal_main + 1765
frame #2: 0x00000001000032d8 seg`main + 40
frame #3: 0x00007fff96d555ad libdyld.dylib`start + 1
from crystal-pg.
I made an rds instance.
crystal-pg β€ lldb test-crash git:master*
(lldb) target create "test-crash"
Current executable set to 'test-crash' (x86_64).
(lldb) r
Process 18919 launched: '/Users/will/code/crystal-pg/test-crash' (x86_64)
Process 18919 stopped
* thread #1: tid = 0x53ab879, 0x00007fff8b8a2a13 libssl.0.9.8.dylib`SSL_set_bio + 19, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
frame #0: 0x00007fff8b8a2a13 libssl.0.9.8.dylib`SSL_set_bio + 19
libssl.0.9.8.dylib`SSL_set_bio:
-> 0x7fff8b8a2a13 <+19>: movq 0x20(%rbx), %rax
0x7fff8b8a2a17 <+23>: testq %rax, %rax
0x7fff8b8a2a1a <+26>: je 0x7fff8b8a2a32 ; <+50>
0x7fff8b8a2a1c <+28>: cmpq %rax, 0x18(%rbx)
(lldb) bt
* thread #1: tid = 0x53ab879, 0x00007fff8b8a2a13 libssl.0.9.8.dylib`SSL_set_bio + 19, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
* frame #0: 0x00007fff8b8a2a13 libssl.0.9.8.dylib`SSL_set_bio + 19
frame #1: 0x0000000100037baf test-crash`*OpenSSL::SSL::Socket#initialize<OpenSSL::SSL::Socket, (OpenSSL::SSL::Socket | Socket+), Symbol, OpenSSL::SSL::Context, Bool>:Int32 + 335
frame #2: 0x0000000100037a49 test-crash`*OpenSSL::SSL::Socket::new:sync_close<(OpenSSL::SSL::Socket | Socket+), Bool>:OpenSSL::SSL::Socket + 137
frame #3: 0x000000010004d1cf test-crash`*PQ::Connection#negotiate_ssl<PQ::Connection>:Nil + 591
frame #4: 0x000000010004cf68 test-crash`*PQ::Connection#initialize<PQ::Connection, PQ::ConnInfo>:Nil + 360
frame #5: 0x000000010004cde9 test-crash`*PQ::Connection::new<PQ::ConnInfo>:PQ::Connection + 233
frame #6: 0x0000000100038ea6 test-crash`*PG::Connection#initialize<PG::Connection, PQ::ConnInfo>:Bool + 70
frame #7: 0x0000000100038e49 test-crash`*PG::Connection#initialize<PG::Connection, String>:Bool + 137
frame #8: 0x0000000100038d9c test-crash`*PG::Connection::new<String>:PG::Connection + 76
frame #9: 0x0000000100038d35 test-crash`*PG::connect<String>:PG::Connection + 21
frame #10: 0x000000010000abeb test-crash`__crystal_main + 41883
frame #11: 0x000000010000f9e8 test-crash`main + 40
frame #12: 0x00007fff96d555ad libdyld.dylib`start + 1
frame #13: 0x00007fff96d555ad libdyld.dylib`start + 1
(lldb)
from crystal-pg.
@will thanks a lot.
Here's lldb stacktrace.
* thread #1: tid = 0x174f35f, 0x00007fff8c639a13 libssl.0.9.8.dylib`SSL_set_bio + 19, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
frame #0: 0x00007fff8c639a13 libssl.0.9.8.dylib`SSL_set_bio + 19
libssl.0.9.8.dylib`SSL_set_bio:
-> 0x7fff8c639a13 <+19>: movq 0x20(%rbx), %rax
0x7fff8c639a17 <+23>: testq %rax, %rax
0x7fff8c639a1a <+26>: je 0x7fff8c639a32 ; <+50>
0x7fff8c639a1c <+28>: cmpq %rax, 0x18(%rbx)
(lldb) bt
* thread #1: tid = 0x174f35f, 0x00007fff8c639a13 libssl.0.9.8.dylib`SSL_set_bio + 19, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x20)
* frame #0: 0x00007fff8c639a13 libssl.0.9.8.dylib`SSL_set_bio + 19
frame #1: 0x000000010003255f sbux-admin`*OpenSSL::SSL::Socket#initialize<OpenSSL::SSL::Socket, (OpenSSL::SSL::Socket | Socket+), Symbol, OpenSSL::SSL::Context, Bool>:Int32 + 335
frame #2: 0x00000001000323f9 sbux-admin`*OpenSSL::SSL::Socket::new:sync_close<(OpenSSL::SSL::Socket | Socket+), Bool>:OpenSSL::SSL::Socket + 137
frame #3: 0x0000000100044f5f sbux-admin`*PQ::Connection#negotiate_ssl<PQ::Connection>:Nil + 591
frame #4: 0x0000000100044cf8 sbux-admin`*PQ::Connection#initialize<PQ::Connection, PQ::ConnInfo>:Nil + 360
frame #5: 0x0000000100044b79 sbux-admin`*PQ::Connection::new<PQ::ConnInfo>:PQ::Connection + 233
frame #6: 0x0000000100033386 sbux-admin`*PG::Connection#initialize<PG::Connection, PQ::ConnInfo>:Bool + 70
frame #7: 0x0000000100033329 sbux-admin`*PG::Connection#initialize<PG::Connection, String>:Bool + 137
frame #8: 0x0000000100033284 sbux-admin`*PG::Connection::new<String>:PG::Connection + 84
frame #9: 0x0000000100033215 sbux-admin`*PG::connect<String>:PG::Connection + 21
frame #10: 0x000000010000ae1b sbux-admin`__crystal_main + 41883
frame #11: 0x000000010000f2a8 sbux-admin`main + 40
frame #12: 0x00007fff8ce3c5ad libdyld.dylib`start + 1
from crystal-pg.
@ssl = LibSSL.ssl_new(context)
is returning Pointer(Void).null
:/ in OpenSSL::SSL::Socket#initialize
from crystal-pg.
Oops. Seems we are missing a null check there.
from crystal-pg.
Also context = #<OpenSSL::SSL::Context:0x101bcdea0 @handle=Pointer(Void).null>
from crystal-pg.
Nice catch π
from crystal-pg.
I guess at that time we wanted to try to see if it works but didn't check for null... in any place π
from crystal-pg.
I'll add the null checks and raise OpenSSL::SSL::Error. Sounds good? (or you can send a PR with this, I'm about to leave for now)
from crystal-pg.
@asterite that sounds good, I won't do a pr, but I donβt know why the handle
ivar is null to begin with. Raising will fix the segfualt, the ssl connection still wont work right?
from crystal-pg.
@will Nope, it won't work. I don't know about openssl and the man doesn't seem to be very helpful: "The creation of a new SSL_CTX object failed. Check the error stack to find out the reason."
from crystal-pg.
Oh that reminds me of a terrible bug @uhoh-itsmaciek found with ruby and the ssl error stack. If you don't clear it after errors it just grows forever or something.
from crystal-pg.
http://www.educatedguesswork.org/2005/03/curse_you_opens.html
What's happening is this: it's a result of the way that OpenSSL handles errors. It maintains a per-thread (static in our case) error stack. When you call SSL_get_error(r,ssl) it combines the information from r,ssl, and the error stack to decide what to return. Now, here's the important point: the error stack isn't cleared automatically on the call to SSL_write().
from crystal-pg.
from crystal-pg.
@asterite meanwhile this issue doesn't exist in Crystal 0.15.0
from crystal-pg.
@sdogruyol interesting. I'll rollback to Crystal 0.15.0 and see if it goes away for me.
I am another datapoint for
0.16.0 = Program exited because of a segmentation fault (11)
rollback to 0.15.0 = works
from crystal-pg.
@datachomp @sdogruyol very interesting. I assume it's the same version of openssl that is being linked each time?
from crystal-pg.
@will i think so. How can we be sure?
from crystal-pg.
on os x otool -L <compiled_program>
from crystal-pg.
Here's the working one with Crystal 0.15.0
/usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
/usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
/usr/local/lib/libevent-2.0.5.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/local/lib/libgc.1.dylib (compatibility version 2.0.0, current version 2.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
The segfault one
/usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
/usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
/usr/local/lib/libevent-2.0.5.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libpcre.0.dylib (compatibility version 1.0.0, current version 1.1.0)
/usr/local/lib/libgc.1.dylib (compatibility version 2.0.0, current version 2.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
from crystal-pg.
Looks like the same libssl, which makes sense.
Looking at the diff in crystal between 0.15 and 0.16, nothing is jumping out to me
crystal β€ git diff 0.15.0..0.16.0 src/openssl/ | pbcopy
diff --git a/src/openssl/bio.cr b/src/openssl/bio.cr
index eb83135..b6fcd8b 100644
--- a/src/openssl/bio.cr
+++ b/src/openssl/bio.cr
@@ -48,11 +48,9 @@ struct OpenSSL::BIO
crystal_bio
end
- @io : IO
@boxed_io : Void*
- @bio : LibCrypto::Bio*
- def initialize(@io)
+ def initialize(@io : IO)
@bio = LibCrypto.bio_new(pointerof(CRYSTAL_BIO))
# We need to store a reference to the box because it's
diff --git a/src/openssl/cipher.cr b/src/openssl/cipher.cr
index 94ea0ad..8f8ecf7 100644
--- a/src/openssl/cipher.cr
+++ b/src/openssl/cipher.cr
@@ -4,8 +4,6 @@ class OpenSSL::Cipher
class Error < OpenSSL::Error
end
- @ctx : Pointer(Void)?
-
def initialize(name)
cipher = LibCrypto.evp_get_cipherbyname name
raise ArgumentError.new "unsupported cipher algorithm #{name.inspect}" unless cipher
diff --git a/src/openssl/digest/digest.cr b/src/openssl/digest/digest.cr
index cf6d390..c627655 100644
--- a/src/openssl/digest/digest.cr
+++ b/src/openssl/digest/digest.cr
@@ -10,7 +10,6 @@ module OpenSSL
include DigestBase
getter name : String
- @ctx : LibCrypto::EVP_MD_CTX
def initialize(@name, @ctx : LibCrypto::EVP_MD_CTX)
raise Error.new("Invalid EVP_MD_CTX") unless @ctx
diff --git a/src/openssl/hmac.cr b/src/openssl/hmac.cr
index 4ab5472..b637b87 100644
--- a/src/openssl/hmac.cr
+++ b/src/openssl/hmac.cr
@@ -1,7 +1,7 @@
require "./lib_crypto"
class OpenSSL::HMAC
- def self.digest(algorithm : Symbol, key, data)
+ def self.digest(algorithm : Symbol, key, data) : Slice(UInt8)
evp = case algorithm
when :dss then LibCrypto.evp_dss
when :dss1 then LibCrypto.evp_dss1
@@ -23,7 +23,7 @@ class OpenSSL::HMAC
buffer[0, buffer_len.to_i]
end
- def self.hexdigest(algorithm : Symbol, key, data)
+ def self.hexdigest(algorithm : Symbol, key, data) : String
digest(algorithm, key, data).hexstring
end
end
diff --git a/src/openssl/md5.cr b/src/openssl/md5.cr
index 7941351..f218907 100644
--- a/src/openssl/md5.cr
+++ b/src/openssl/md5.cr
@@ -1,11 +1,11 @@
require "./lib_crypto"
class OpenSSL::MD5
- def self.hash(data : String)
+ def self.hash(data : String) : UInt8[16]
hash(data.to_unsafe, data.bytesize)
end
- def self.hash(data : UInt8*, bytesize : Int)
+ def self.hash(data : UInt8*, bytesize : Int) : UInt8[16]
buffer = uninitialized UInt8[16]
LibCrypto.md5(data, bytesize, buffer)
buffer
diff --git a/src/openssl/pkcs5.cr b/src/openssl/pkcs5.cr
index a25cc3e..06a8df5 100644
--- a/src/openssl/pkcs5.cr
+++ b/src/openssl/pkcs5.cr
@@ -1,7 +1,7 @@
require "./openssl"
module OpenSSL::PKCS5
- def self.pbkdf2_hmac_sha1(secret, salt, iterations = 2**16, key_size = 64)
+ def self.pbkdf2_hmac_sha1(secret, salt, iterations = 2**16, key_size = 64) : Slice(UInt8)
buffer = Slice(UInt8).new(key_size)
if LibCrypto.pkcs5_pbkdf2_hmac_sha1(secret, secret.bytesize, salt, salt.bytesize, iterations, key_size, buffer) != 1
raise OpenSSL::Error.new "pkcs5_pbkdf2_hmac"
diff --git a/src/openssl/ssl/context.cr b/src/openssl/ssl/context.cr
index 80d707b..481b069 100644
--- a/src/openssl/ssl/context.cr
+++ b/src/openssl/ssl/context.cr
@@ -1,10 +1,9 @@
class OpenSSL::SSL::Context
- @@default : OpenSSL::SSL::Context?
-
- def self.default
+ def self.default : self
@@default ||= new
end
+ # Do not remove this until version > 0.15.0, it's needed in 0.15.0
@handle : LibSSL::SSLContext
def initialize
diff --git a/src/openssl/ssl/socket.cr b/src/openssl/ssl/socket.cr
index 0da517f..9cca718 100644
--- a/src/openssl/ssl/socket.cr
+++ b/src/openssl/ssl/socket.cr
@@ -1,9 +1,6 @@
class OpenSSL::SSL::Socket
include IO
- @ssl : LibSSL::SSL
- @bio : OpenSSL::BIO
-
# If `sync_close` is true, closing this socket will
# close the underlying IO.
property? sync_close : Bool
from crystal-pg.
How about /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
?
from crystal-pg.
That's for gzip, I donβt think that'd cause anything seen here.
from crystal-pg.
Yeah. The diff is also looking fine. Weird
from crystal-pg.
@sdogruyol @will If you want you can email me any data so I can try to reproduce this and see what's wrong
from crystal-pg.
I'll send you creds to a db that reproduces this tomorrow
On Monday, May 16, 2016, Ary Borenszweig [email protected] wrote:
@sdogruyol https://github.com/sdogruyol @will https://github.com/will
If you want you can email me any data so I can try to reproduce this and
see what's wrongβ
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/will/crystal-pg/issues/40#issuecomment-219589529
from crystal-pg.
So, it turned out that the segfault only happens if you create a connection in a constant. The problem is that these four lines are considered main code, and main code is executed before constants (and class vars) are initialized.
I don't know what's the general solution to this is, but for now I pushed a workaround. I'll probably release 0.17.1 tomorrow with this fix.
from crystal-pg.
Just tried with 0.17.1 and it works beautifully. Thank you so much to everyone involved!
from crystal-pg.
0.17.1 is out and 0.17.3 will be out soon with another fix, so I'm going to close this now. Thanks for reporting!
from crystal-pg.
Related Issues (20)
- can't find file 'crypto/md5' Using pg (0.11.0) HOT 1
- shards update/install/build failing at the crystal-pg step HOT 2
- What versions are supported? HOT 2
- Unhandled exception: cannot insert multiple commands into a prepared statement (PQ::PQError) HOT 3
- getting OverflowError:Arithmetic overflow HOT 6
- Error: no overload matches 'PG::Decoders.array_decoder' with type UUID.class HOT 1
- Connecting to unix socket with url HOT 3
- Shards update broken on Crystal 1.0 HOT 2
- Too many successive queries result in DB::ConnectionLost HOT 4
- Unable to connect to Cockroachdb HOT 2
- New Release 0.23.3? HOT 2
- cockroach db HOT 2
- Proper implementation of `sslmode=verify-full` HOT 2
- Unhandled exception: column "foo" does not exist (PQ::PQError) - How to insert into table without specifying all the columns HOT 1
- Time seems to drop precision when passed in as an arg using at_end_of_day HOT 5
- Tables with upcase symbols not readed HOT 2
- Error: can't cast to JSON::Any
- Add ability to automatically return `TIMESTAMPTZ`->`Time` in the Postgres session's time zone
- Exception sending query with bytea[] binary array type? HOT 2
- Support inserting `PG::Interval` instances HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from crystal-pg.