Truffle slower than MRI in standard lib Queue about truffleruby HOT 10 CLOSED

@chrisseaton The benchmark-ips version is single-threaded, while the original benchmark has 2 working threads.
This probably means under no contention we perform well, but under higher contention not so much since the current implementation uses a single monitor:
https://github.com/graalvm/truffleruby/blob/44449275a63a3640339a70c8a169b33487def69c/src/main/java/org/truffleruby/core/queue/UnsizedQueue.java#L20

from truffleruby.

Thanks very much for the report - I'll look into this.

from truffleruby.

I reduced this to an initial simpler benchmark and can reproduce that it's slower.

require 'benchmark/ips'

queue = Queue.new

Benchmark.ips do |x|

  x.report('push-pop') do
    queue << 1
    queue.pop
  end

end

MRI 2.3.5

Warming up --------------------------------------
            push-pop   230.218k i/100ms
Calculating -------------------------------------
            push-pop      5.972M (± 5.3%) i/s -     29.928M in   5.025320s

JRuby 9.1.13.0

Warming up --------------------------------------
            push-pop   183.793k i/100ms
Calculating -------------------------------------
            push-pop      6.651M (± 9.2%) i/s -     32.899M in   4.992694s

TruffleRuby

Warming up --------------------------------------
            push-pop    40.255k i/100ms
Calculating -------------------------------------
            push-pop      1.054M (± 6.9%) i/s -      5.233M in   4.992351s

from truffleruby.

Oh sorry my mistake - I had Graal misconfigured. This is what I see for TruffleRuby.

Warming up --------------------------------------
            push-pop   117.103k i/100ms
Calculating -------------------------------------
            push-pop     11.527M (±16.2%) i/s -     51.174M in   4.988282s

Which is around 2x faster than MRI or JRuby.

But your code is still around 30% slower as you say, so there's something else wrong.

from truffleruby.

That does seem to make the difference. But does MRI use anything more sophisticated than a single simple lock? I'll need to look at what JRuby do.

require 'benchmark/ips'

queue = Queue.new

producer = Thread.new do
  while true do
    next if queue.size > 250000
    queue << 'hello'
  end
end

Benchmark.ips do |x|

  x.report('push-pop') do
    queue.pop
  end

end

MRI

Warming up --------------------------------------
            push-pop   170.941k i/100ms
Calculating -------------------------------------
            push-pop      4.138M (±81.0%) i/s -     12.479M in   5.043374s

JRuby

Warming up --------------------------------------
            push-pop    77.158k i/100ms
Calculating -------------------------------------
            push-pop      3.363M (±25.0%) i/s -     13.734M in   4.963765s

TruffleRuby

Warming up --------------------------------------
            push-pop    46.241k i/100ms
Calculating -------------------------------------
            push-pop      1.388M (±61.0%) i/s -      6.243M in   5.001231s

The loop for the original code up (ignoring the uncommon case where the loop needs to print) looks good as well, so that also suggests that yes it's the contention.

from truffleruby.

@chrisseaton @eregon thanks a lot for looking into this

from truffleruby.

JRuby uses a variant of the LinkedBlockingQueue, which uses 2 locks, putLock and takeLock, so pop and push are not contending if there are enough elements in between such as in this benchmark.
https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/ext/thread/Queue.java

Recent MRI seems to just use the GIL to protect the Queue and a normal Ruby Array is used underneath as the storage:
https://github.com/ruby/ruby/blob/6b05153a3a75b74b64553d6a46f501d9ee0f0376/thread_sync.c#L713-L721
They use an internal linked list (from ccan/list) to handle blocking and waking up threads.

@prdn As a side note, a SizedQueue might be better in this case where you want to block the producer to be too much ahead.

from truffleruby.

@eregon thanks for the suggestion. I'll definitely try it.
jRuby solution makes sense

from truffleruby.

Running with version truffleruby RC10 seems to run faster:

MRI ruby-2.5.1:

Warming up --------------------------------------
            push-pop   308.053k i/100ms
Calculating -------------------------------------
            push-pop      7.439M (± 0.8%) i/s -     37.274M in   5.010944s

truffleruby-1.0.0-rc10:

Warming up --------------------------------------
            push-pop   573.244k i/100ms
Calculating -------------------------------------
            push-pop      9.037M (±13.8%) i/s -     44.140M in   5.057861s

from truffleruby.

For me on 19.1 the original benchmark now runs as fast as MRI in CE, and is faster in EE. JRuby is still faster but I'll close the issue as it's now not slower than MRI.

from truffleruby.