Giter VIP home page Giter VIP logo

Comments (3)

arvidn avatar arvidn commented on May 27, 2024

the python callback requires that libtorrent locks the GIL, which is questional to begin with. This is probably the reason for the deadlock when using multiple threads.

In your example, you just return True anyway. If you don't specify it, you will have the same behavior, but without the need to lock the GIL

from libtorrent.

categorical avatar categorical commented on May 27, 2024

Thanks @arvidn for the reply. Yes there are ways to circumvent this and I did, though "not specify it" is signature mismatch from python side. I used to have production servers all stall every few hours and days and without knowing why had to script to reset them, and now I can confirm this deadlock is indeed the casue.

fyi, regarding the deadlock, what I see is that say there are 3 threads of concern: a) python, b) python and c) session,
then a) calls get_torrent_status and c) calls callback but released gil mid way, if b) acquired gil now it waits on c) and c) waits on gil. I tried some changes and the deadlock goes away from my settings.

v1.2.19

--- a/bindings/python/src/session.cpp
+++ b/bindings/python/src/session.cpp
@@ -27,6 +27,8 @@
 #include <libtorrent/extensions/smart_ban.hpp>
 #include <libtorrent/extensions/ut_metadata.hpp>
 #include <libtorrent/extensions/ut_pex.hpp>
+#include <sstream>
+#include <sys/syscall.h>
 
 namespace boost
 {
@@ -521,6 +523,11 @@ namespace
 
     bool wrap_pred(object pred, torrent_status const& st)
     {
+        long int tid=syscall(SYS_gettid);
+        fprintf(stdout,"thread %-10ld gil %d\n",tid,PyGILState_Check());
+
+        PyEval_InitThreads();
+        lock_gil g; 
         return pred(st);
     }
 
@@ -531,12 +538,14 @@ namespace
         // libtorrent thread the python predicate will be freed from that
         // thread, which won't work
         auto wrapped_pred = std::bind(&wrap_pred, pred, std::placeholders::_1);
+        list ret;
+        Py_BEGIN_ALLOW_THREADS
         std::vector<torrent_status> torrents
             = s.get_torrent_status(std::ref(wrapped_pred), status_flags_t(flags));
 
-        list ret;
         for (std::vector<torrent_status>::iterator i = torrents.begin(); i != torrents.end(); ++i)
             ret.append(*i);
+        Py_END_ALLOW_THREADS
         return ret;
     }

from libtorrent.

categorical avatar categorical commented on May 27, 2024

ways to circumvent are use refresh_torrent_status instead or by not calling in threads at all:)

from libtorrent.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.