"this" corrupted about qtbase HOT 14 CLOSED

Looked at it for a few hours this morning; no solution yet. When R
evaluates "this" it allocates an environment that uses the same object
table as the environment enclosing the paintEvent handler. My working
theory is that if R finalizes "this", somehow the object table also gets
finalized, so the enclosing environment of the handler is no longer valid.
Not sure if the R implementation expects two environments to share an
object table. That would not ordinarily happen, but it should work in
theory. We can't just use the same environment, because the parent
environment is different.

On Fri, Jun 6, 2014 at 4:45 AM, Tomas Sieger [email protected]
wrote:

"this" gets corrupted in user's methods overriding Qt widgets methods
(like the paintEvent method below) running the 'qt4' branch and Qt4:

qsetClass("Label", Qt$QLabel, function(parent = NULL) {
super(parent)
})

qsetMethod("paintEvent", Label, function(event) {
print(this)
super("paintEvent",event)
},"protected")

window <- Qt$QWidget()
queryLabel1 <- Label("label 1")
queryLabel2 <- Label("label 2")
queryLabel3 <- Label("label 3")

layout <- Qt$QHBoxLayout()
layout$addWidget(queryLabel1)
layout$addWidget(queryLabel2)
layout$addWidget(queryLabel3)

window$setLayout(layout)
window$show()

When resizing the window for a while (few seconds at most), or simply
focusing and defocusing the window, R crashes saying:

*** caught segfault ***
address 0x10, cause 'memory not mapped'

Traceback:
1: print(this)
2: (function (event) { print(this) super("paintEvent", event)})()

The problem is neither related to creating several instances of the same
class, nor to QtLabel.

Reproducible on Ubuntu 14.04 on Intel Core 2 Duo, both 64-bit and 32-bit,
but also on Mac OS X 10.8.4, Mac Pro Pro 5.1, 6-Core Intel Xeon 2.4GHz, but
not on Fedora Core Linux 3.6.11-1.fc17.i686.PAE i686.

sessionInfo:
R version 3.1.0 (2014-04-10)
Platform: x86_64-pc-linux-gnu (64-bit)

attached base packages:
[1] stats graphics grDevices utils datasets methods base qtbase_1.0.6

Not sure whether the same problem appears using the main qtbase branch and
Qt5.2+ (can't build qtbase for some reason - working on it).

Any ideas are welcome.

—
Reply to this email directly or view it on GitHub
#21.

from qtbase.

Thanks for sharing your theory.
Indeed, the problem is not only with "this", but perhaps with the (parent) environment of the handler - even if I do not touch "this" in the handler, but only print something, after a while the handler crashes, e.g. when flushing stdout (!):

 *** caught segfault ***
address 0x10, cause 'memory not mapped'

Traceback:
 1: flush(stdout())
 2: (function (event) {    print("paintEvent3")    flush(stdout())    print("leaving paintEvent3")    flush(stdout())})(<environment>)

At the same time, the bug seems to be (quite logically) related to global memory conditions (memory allocation, or possibly garbage collection?). When I inspect "this" using .Internal(inspect(this)) in the handler, I can see the address of "this" is different on each call to the handler - it keeps gradually changing from say <0x25cb908> thru <0x2610de0> up to
<0x32f6620> and then, once this gets (re)allocated "lower" in the address space, e.g.
<0x1c1a520>, R crashes. Interestingly, it crashes not in the handler in which "this" is being inspected, but another one - so the problem is more global. (Perhaps a trivial observation.)

from qtbase.

Thanks for looking into this more deeply. Yes, a new environment gets
allocated each time, but it shares the same symbol table as the parent
environment of the handler frame. The allocation to a lower address is
probably after the GC has run (and invalidated things inappropriately). I
don't get why R finalizes the symbol table (an externalptr), despite it
being referenced by the handler frame itself. I have noticed that if I
invoke gc() in the handler, the crashes go away, and the garbage collector
behaves correctly. So it's something about references being left around
over multiple invocations of the handler... but it's tough to pin-down. I
even tried running gc() randomly in the handler, e.g., ~ every 50
invocations, and everything kept working. I'm guessing that if gc() were
invoked much less frequently, R would end up invoking it, perhaps from a
different context and maybe that makes a difference? Starting to smell like
a bug/limitation in R...

On Wed, Jun 11, 2014 at 12:28 PM, Tomas Sieger [email protected]
wrote:

Thanks for sharing your theory.
Indeed, the problem is not only with "this", but perhaps with the (parent)
environment of the handler - even if I do not touch "this" in the handler,
but only print something, after a while the handler crashes, e.g. when
flushing stdout (!):

*** caught segfault ***
address 0x10, cause 'memory not mapped'

Traceback:
1: flush(stdout())
2: (function (event) { print("paintEvent3") flush(stdout()) print("leaving paintEvent3") flush(stdout())})()

At the same time, the bug seems to be (quite logically) related to global
memory conditions (memory allocation, or possibly garbage collection?).
When I inspect "this" using .Internal(inspect(this)) in the handler, I
can see the address of "this" is different on each call to the handler - it
keeps gradually changing from say <0x25cb908> thru <0x2610de0> up to
<0x32f6620> and then, once this gets (re)allocated "lower" in the address
space, e.g.
<0x1c1a520>, R crashes. Interestingly, it crashes not in the handler in
which "this" is being inspected, but another one - so the problem is more
global. (Perhaps a trivial observation.)

—
Reply to this email directly or view it on GitHub
#21 (comment).

from qtbase.

Thanks for the further investigation and explanation. The trick with the explicit calls to GC seems to work, however, I'm experiencing that the GC must be called much more often than on each 50th call to the handler, which makes my app terribly slow :-(. Also, the exact GC calling frequency is probably system-dependent and hard to guess in advance.

In the meantime, I've tried to workaround the problem by bypassing the usage of the "bad" environments in the handler by running the handler in my own environment holding only the "event" parameter (and having the global env as the parent), and replacing "this" with a globally visible singleton (see below), but that still does not work, perhaps because the "bad" environments get still constructed and used during the call to the handler (even though the handler does not touch them).

myenv<-function(nm,val){e<-new.env(globalenv());assign(nm,val,e);e}

f1<-function(event) {
evalq({
  qtbase:::qinvokeSuper(label1,'paintEvent',event)
},myenv('event',event),globalenv())
}
f2<-function(event) {
evalq({
  qtbase:::qinvokeSuper(label2,'paintEvent',event)
},myenv('event',event),globalenv())
}

qsetClass("Label1", Qt$QLabel, function(parent = NULL) {
  super(parent)
})
qsetMethod("paintEvent", Label1, f1,"protected")

qsetClass("Label2", Qt$QLabel, function(parent = NULL) {
  super(parent)
})
qsetMethod("paintEvent", Label2, f2,"protected")

window <- Qt$QWidget()
label1 <- Label1("label 1")
label2 <- Label2("label 2")

layout <- Qt$QHBoxLayout()
layout$addWidget(label1)
layout$addWidget(label2)

window$setLayout(layout)
window$show()

BTW provided your theory is true (and pardon my limited knowledge) - would not be it possible to solve the problem by not sharing the object table between "this" and the environment enclosing the paintEvent handler, but copying the table (plus implementing some write-thru mechanism, if necessary)?

from qtbase.

Update: the issue demonstrates only in R3.1.0+ (not in previous R versions, even R 3.0.3).
It still appears under patched (3.1.0 Patched (2014-06-12 r65926)) and development (R Under development (unstable) (2014-06-12 r65926)) R versions.

BTW I've noticed that NEWS to R 3.1.0 read: "Also, a number of other changes to reduce copying of objects; all contributed by or based on suggestions by Michael Lawrence." :-)

from qtbase.

This is an interesting finding, thanks. My changes were to the
implementation of copy-on-write; pretty much unrelated to the garbage
collector. But maybe we should bring this to the attention of Luke?

On Fri, Jun 13, 2014 at 7:46 AM, Tomas Sieger [email protected]
wrote:

Update: the issue demonstrates only in R3.1.0+ (not in previous R
versions, even R 3.0.3).
It still appears under patched (3.1.0 Patched (2014-06-12 r65926)) and
development (R Under development (unstable) (2014-06-12 r65926)) R
versions.

BTW I've noticed that NEWS to R 3.1.0 read: "Also, a number of other
changes to reduce copying of objects; all contributed by or based on
suggestions by Michael Lawrence." :-)

—
Reply to this email directly or view it on GitHub
#21 (comment).

from qtbase.

That's a good idea. Could you do it, please?

from qtbase.

Luke says that it is very unlikely to be an issue with the GC. He says that it must just be a slight change in memory allocation behavior. That may be the case; however, I've scoured the code for any missing PROTECT()s and came up empty. I just don't understand how the parent of the current frame of evaluation does not have a protected object table.

from qtbase.

I investigated it more deeply and found out the problem is that the object table gets finalized as the environment it is associated to is considered unreachable in finalization (even though the object table is in use at that time in another environment).

I've injected R and qtbase with some debugs and mention some output here for reference:

key:
a0x45977a8 - R_ObjectTable instance
a0x66765e8 - SEXP R external pointer wrapper for a0x45977a8
a0x5d728b8 - WeakRef for a0x66765e8
a0x66765b0 - SEXP environmnent (perhaps of the "paintEvent" handler)
a0x55de298 - SEXP environmnent (perhaps of another invocation of the "paintEvent" handler)
a0x415e370 - parent environment of the two environments above

graphically:
                          +- a0x66765e8 -+   
a0x66765b0 -(hashtab)-->  |   wrapped:   |  <-(WeakRef)- a0x5d728b8
a0x55de298 -a0x5d728b8->  |  a0x45977a8  |
                          +--------------+

log (the "story" follows):
01 [qtbase:SmokeObject::enclose() this a0xfcd6fc0:] calling internalSexp
02 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling createSexp for env a0x415e370
03 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:]
04 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:] setting enclosing env a0x415e370 for env a0x66765b0
05 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling internalTable
06 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] called
07 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] calling _klass->createObjectTable(this)
08 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] creating sexp for hashtab a0xfcd7730
09 [qtbase:ObjectTable::createSexp this a0xfcd7730:] wrapPointer(tb a0x45977a8)
10 [qtbase:wrapPointer:] wrapping a0x45977a8 into sexp a0x66765e8
11   [WEAK REF CHECK:] sexp a0x5d728b8 not ready, not marked
12 [R_MakeWeakRefC:] making weakRef a0x5d728b8 for key a0x66765e8
13 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] internal table sexp a0x66765e8 created for hashtab a0xfcd7730
14 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] setting hashtab a0x66765e8 to sexp a0x66765b0
15 [envir.c: findFun] checkEnv: rho a0x66765b0, table a0x45977a8
16 findVarInFrame3 rho a0x66765b0, symbol {, table a0x45977a8
17 findVarInFrame3 rho a0x66765b0, symbol {, table a0x45977a8: a0x40a156c

18 [CheckFinalizers:] setting a0x5d728b8 as ready to finalize (was not ready, not marked)
19 [FORWARD_NODE:] marking node a0x5d728b8
20 [FORWARD_NODE:] marking node a0x66765e8
21  [WEAK REF CHECK:]sexp a0x5d728b8 ready, marked

22 [qtbase:SmokeObject::enclose() this a0xfcd6fc0:] calling internalSexp
23 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling createSexp for env a0x415e370
24 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:]
25 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:] setting enclosing env a0x415e370 for env a0x55de298
26 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling internalTable
27 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] called
28 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] setting hashtab a0x66765e8 to sexp a0x55de298
29 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
30 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8
31 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8: a0x40a156c
32 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
33 findVarInFrame3 rho a0x55de298, symbol if, table a0x45977a8
34 findVarInFrame3 rho a0x55de298, symbol if, table a0x45977a8: a0x40a156c
35 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
36 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8
37 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8: a0x40a156c
38 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
39 findVarInFrame3 rho a0x55de298, symbol cat, table a0x45977a8
40 findVarInFrame3 rho a0x55de298, symbol cat, table a0x45977a8: a0x40a156c
41 findVarInFrame3 rho a0x55de298, symbol print.ls_str, table a0x45977a8
42 findVarInFrame3 rho a0x55de298, symbol print.ls_str, table a0x45977a8: a0x40a156c
43  [WEAK REF CHECK:] sexp a0x5d728b8 ready, marked
44 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a0x45977a8
45 [qtbase:ObjectTable::~ObjectTable() this=a0xfe99fd0] extptr a0x6d5961c, tb a0xfe9a010
46 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a0x45977a8
47 about to run R_RunWeakRefFinalizer a0x5d728b8 (saved rho a0x6722bcc)
48 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a0x45977a8
49 [qtbaseLInstanceObjectTable::~InstanceObjectTable() this a0xfcd7730] sexp a0x66765e8
50 [qtbase:ObjectTable::~ObjectTable() this=a0xfcd7730] extptr a0x66765e8, tb a0x45977a8
51 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
52 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a(nil)
53 [qtbase:ObjectTable::~ObjectTable() this=a0x45133e0] extptr a0x52f2348, tb a0x4513420
54 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
55 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a(nil)
56 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
57 [envir.c: findFun] checkEnv: rho a0x55de298, table a(nil)
58 findVarInFrame3 rho a0x55de298, symbol cat, table a(nil)
59 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)

The problem is that the handler environment a0x66765b0, which is associated with the object table a0x45977a8 (lines 09-14) seems to get lost somehow (?), i.e. not seen anymore (for the last time at line 17). As a finalizer has been associated with the object table a0x45977a8 thru the "lost" environment a0x66765b0 (line 12), and the environment can't be reached in CheckFinalizers() (just guessing, no direct evidence), the weak ref is set as "ready to be finalized" (line 18). Then, the object table gets reused in another environment a0x55de298 (lines 22-28) and used heavily (lines 29-42). However, even though the object table a0x45977a8 is in use, it is finalized (lines 49-50) along with its wrapping external pointer a0x66765e8.

I do not know the reason why the environment a0x66765b0 disappears (or simply can't be reached). I'd happilly go deeper in it, but now I don't see where and how to look. I can also provide my hacked R/qtbase sources, if you like.

As a quick, dirty (and incorrect!) workaround, I tried to instantiate a new object table on each request (see https://github.com/tsieger/qtbase/tree/object_table_copy), which seems
to work (no more crashing). However, it destroys the sharing of the object table, so it is not a proper solution.

Would it be possible to share the object table and enable proper finalization at the same time?
I thought of some reference counting mechanism, but I don't know how to make callers release the refcounts. On the other hand, instantiating a new table on each request and delegating operations to some shared table would also be problematic - how to housekeep that shared table? (I don't understand why smoke object can't housekeep the object table, and just orphans it? Maybe, the answer would invalidate the shared object table idea.)

Would you prefer to investigate the cause of the problem (i.e. in R, how?), or adapt qtbase?

from qtbase.

Thanks for this in-depth analysis. I really wish I understood the problem.
Perhaps it is that R decides it will free an object prior to actually
calling the finalizer. This would cause qtbase to reuse the table without
knowing that it was a "dead man walking".

If that is indeed the case, then it means that qtbase has made invalid
assumptions all over the place, and it would be tricky to adapt the design.
But I'm pretty sure that's not the case.

On Sun, Jun 22, 2014 at 12:16 PM, Tomas Sieger [email protected]
wrote:

I investigated it more deeply and found out the problem is that the object
table gets finalized as the environment it is associated to is considered
unreachable in finalization (even though the object table is in use at that
time in another environment).

I've injected R and qtbase with some debugs and mention some output here
for reference:

key:
a0x45977a8 - R_ObjectTable instance
a0x66765e8 - SEXP R external pointer wrapper for a0x45977a8
a0x5d728b8 - WeakRef for a0x66765e8
a0x66765b0 - SEXP environmnent (perhaps of the "paintEvent" handler)
a0x55de298 - SEXP environmnent (perhaps of another invocation of the "paintEvent" handler)
a0x415e370 - parent environment of the two environments above

graphically:
+- a0x66765e8 -+
a0x66765b0 -(hashtab)--> | wrapped: | <-(WeakRef)- a0x5d728b8
a0x55de298 -a0x5d728b8-> | a0x45977a8 |
+--------------+

log (the "story" follows):
01 [qtbase:SmokeObject::enclose() this a0xfcd6fc0:] calling internalSexp
02 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling createSexp for env a0x415e370
03 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:]
04 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:] setting enclosing env a0x415e370 for env a0x66765b0
05 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling internalTable
06 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] called
07 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] calling _klass->createObjectTable(this)
08 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] creating sexp for hashtab a0xfcd7730
09 [qtbase:ObjectTable::createSexp this a0xfcd7730:] wrapPointer(tb a0x45977a8)
10 [qtbase:wrapPointer:] wrapping a0x45977a8 into sexp a0x66765e8
11 sexp a0x5d728b8 not ready, not marked
12 [R_MakeWeakRefC:] making weakRef a0x5d728b8 for key a0x66765e8
13 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] internal table sexp a0x66765e8 created for hashtab a0xfcd7730
14 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] setting hashtab a0x66765e8 to sexp a0x66765b0
15 [envir.c: findFun] checkEnv: rho a0x66765b0, table a0x45977a8
16 findVarInFrame3 rho a0x66765b0, symbol {, table a0x45977a8
17 findVarInFrame3 rho a0x66765b0, symbol {, table a0x45977a8: a0x40a156c

18 [CheckFinalizers:] setting a0x5d728b8 as ready to finalize (was not ready, not marked)
19 [FORWARD_NODE:] marking node a0x5d728b8
20 [FORWARD_NODE:] marking node a0x66765e8
21 sexp a0x5d728b8 ready, marked

22 [qtbase:SmokeObject::enclose() this a0xfcd6fc0:] calling internalSexp
23 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling createSexp for env a0x415e370
24 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:]
25 [qtbase:SmokeObject::createSexp() this a0xfcd6fc0:] setting enclosing env a0x415e370 for env a0x55de298
26 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] calling internalTable
27 [qtbase:SmokeObject::internalTable() this a0xfcd6fc0:] called
28 [qtbase:SmokeObject::internalSexp() this a0xfcd6fc0:] setting hashtab a0x66765e8 to sexp a0x55de298
29 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
30 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8
31 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8: a0x40a156c
32 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
33 findVarInFrame3 rho a0x55de298, symbol if, table a0x45977a8
34 findVarInFrame3 rho a0x55de298, symbol if, table a0x45977a8: a0x40a156c
35 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
36 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8
37 findVarInFrame3 rho a0x55de298, symbol {, table a0x45977a8: a0x40a156c
38 [envir.c: findFun] checkEnv: rho a0x55de298, table a0x45977a8
39 findVarInFrame3 rho a0x55de298, symbol cat, table a0x45977a8
40 findVarInFrame3 rho a0x55de298, symbol cat, table a0x45977a8: a0x40a156c
41 findVarInFrame3 rho a0x55de298, symbol print.ls_str, table a0x45977a8
42 findVarInFrame3 rho a0x55de298, symbol print.ls_str, table a0x45977a8: a0x40a156c
43 sexp a0x5d728b8 ready, marked
44 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a0x45977a8
45 [qtbase:ObjectTable::~ObjectTable() this=a0xfe99fd0] extptr a0x6d5961c, tb a0xfe9a010
46 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a0x45977a8
47 about to run R_RunWeakRefFinalizer a0x5d728b8 (saved rho a0x6722bcc)
48 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a0x45977a8
49 [qtbaseLInstanceObjectTable::~InstanceObjectTable() this a0xfcd7730] sexp a0x66765e8
50 [qtbase:ObjectTable::~ObjectTable() this=a0xfcd7730] extptr a0x66765e8, tb a0x45977a8
51 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
52 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a(nil)
53 [qtbase:ObjectTable::~ObjectTable() this=a0x45133e0] extptr a0x52f2348, tb a0x4513420
54 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
55 [R_RunWeakRefFinalizer(ts_rho2)] checkEnv: rho a0x55de298, table a(nil)
56 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)
57 [envir.c: findFun] checkEnv: rho a0x55de298, table a(nil)
58 findVarInFrame3 rho a0x55de298, symbol cat, table a(nil)
59 [R_RunWeakRefFinalizer(ts_rho2)-post1] checkEnv: rho a0x55de298, table a(nil)

The problem is that the handler environment a0x66765b0, which is
associated with the object table a0x45977a8 (lines 09-14) seems to get lost
somehow (?), i.e. not seen anymore (for the last time at line 17). As a
finalizer has been associated with the object table a0x45977a8 thru the
"lost" environment a0x66765b0 (line 12), and the environment can't be
reached in CheckFinalizers() (just guessing, no direct evidence), the weak
ref is set as "ready to be finalized" (line 18). Then, the object table
gets reused in another environment a0x55de298 (lines 22-28) and used
heavily (lines 29-42). However, even though the object table a0x45977a8 is
in use, it is finalized (lines 49-50) along with its wrapping external
pointer a0x66765e8.

I do not know the reason why the environment a0x66765b0 disappears (or
simply can't be reached). I'd happilly go deeper in it, but now I don't see
where and how to look. I can also provide my hacked R/qtbase sources, if
you like.

As a quick, dirty (and incorrect!) workaround, I tried to instantiate a
new object table on each request (see
https://github.com/tsieger/qtbase/tree/object_table_copy), which seems
to work (no more crashing). However, it destroys the sharing of the object
table, so it is not a proper solution.

Would it be possible to share the object table and enable proper
finalization at the same time?
I thought of some reference counting mechanism, but I don't know how to
make callers release the refcounts. On the other hand, instantiating a new
table on each request and delegating operations to some shared table would
also be problematic - how to housekeep that shared table? (I don't
understand why smoke object can't housekeep the object table, and just
orphans it? Maybe, the answer would invalidate the shared object table
idea.)

Would you prefer to investigate the cause of the problem (i.e. in R,
how?), or adapt qtbase?

—
Reply to this email directly or view it on GitHub
#21 (comment).

from qtbase.

qtbase resuses the table because it assumes R still references it, which is definitely not true in R 3.1.0, in which the table gets finalized for some unknown reason.

It seems that my hack (https://github.com/tsieger/qtbase/tree/object_table_copy) is not incorrect - even though I create a new table on each request, all the tables act through the shared smoke object. This solution does not seem to leak, while the memory overhead is relatively small.
Would you please consider to accept this hack as a (temporary) solution (until a better one appears)? I understand that it would not be nice from the design perspective, but I don't see another option to make qtbase working on R 3.1.0, which is, unfortunately, the most commonly used version nowadays.

from qtbase.

Your fix should work and we very much appreciate your help. The main
concern is that we do not understand why this is happening, so there could
be situations where the external "_sexp" falls victim, as well, and that
would require broader changes.

Please go ahead and submit a pull request. You might consider changing the
QList of references to a QSet, just because it seems more semantically
appropriate.

On Fri, Jun 27, 2014 at 12:08 AM, Tomas Sieger [email protected]
wrote:

qtbase resuses the table because it assumes R still references it, which
is definitely not true in R 3.1.0, in which the table gets finalized for
some unknown reason.

It seems that my hack (
https://github.com/tsieger/qtbase/tree/object_table_copy) is not
incorrect - even though I create a new table on each request, all the
tables act through the shared smoke object. This solution does not seem to
leak, while the memory overhead is relatively small.
Would you please consider to accept this hack as a (temporary) solution
(until a better one appears)? I understand that it would not be nice from
the design perspective, but I don't see another option to make qtbase
working on R 3.1.0, which is, unfortunately, the most commonly used version
nowadays.

—
Reply to this email directly or view it on GitHub
#21 (comment).

from qtbase.

Thank you.
I replaced QList with QSet (thanks for the notice; QList was just a relic) and have submitted two pull requests (one for the master, one for the qt4 branch).

Hopefully, more light will be shed on this issue some day and this rather ad-hoc hack will become unnecessary.

from qtbase.

Pull requests have been merged. At this point, we should probably make a release of the 5.x branch to CRAN. Hopefully we get to the bottom of this some day.

from qtbase.