Giter VIP home page Giter VIP logo

Comments (16)

tcalmant avatar tcalmant commented on September 22, 2024

Hi Angelo,
So the idea would be to have something like a use_ipopo_async inheriting the contextlib.asynccontextmanager from Python 3.7, am I right ?
(I'm not used to asyncio yet, I just used it a couple of times on snippets)

I was wondering for some time if I should do a specific version of iPOPO supporting only Python 3.6+ (with enums, typing, ...) but left it aside as it wouldn't bring that much benefits.
In fact, I would have renamed the current master branch to v1, with a 1.0 release of the iPOPO package on PyPI.
The new master would be for 2.x releases.
That being said, the addition of async capabilities would make the 3.6+ (or 3.7+) branch more interesting to work on.

Note that it must be not change the behavior described in OSGi: bundle start/stop and components in/validation must remain blocking to ensure we don't have side effects.

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

I think It's better using python >3.7 because the API of asyncio of python 3.7 are more performing than python 3.6 and also easier to use. The behavior described in the OSGi will not change, what I have on my mind is modifying the framework low level making it asynchronous with asyncio leaving unchanged the high level API, for example the EventAdmin will have the asynchronous method post() and the synchronous method send() which under the hood will be asynchronous and handled by asyncio. What do you think?

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

I think that would be a good idea!
Some parts of the event handling and module loading of Pelix/iPOPO good benefit from that approach as well.

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

Perfect. How can we organize to develop it? If you're interested I've studied asyncio using this book
asyncio Recipes

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

I think it could be changed every part of the code that has blocking IO with an asynchronous one, e.g. replacing the http request with aiohttp. I want to try to reduce the number of Threads of iPOPO making the asyncio EventLoop (the one running in the main Thread) handle the whole framework, the IO task and short time running task. The long running and the blocking code will be scheduled by the main event loop on other new EventLoop running in workers asynchronous thread that will close by themselves when they finish their job. I forgot to tell you that asyncio in python 3.7+ has also new API that are the new standard of asyncio so we'll have to use the suggested asyncio API in python 3.7+. I hope I've explain myself well. Let me know.
PS: here there are all the code snippets of the book that I've posted before but their explanation is only on the book.
https://github.com/Apress/asyncio-recipes

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

So the plan, so far, would be:

  1. Create a v1 branch from the current state of master and release iPOPO 1.0.0 (I have to talk to some users before that)
  2. Change the README, setup.py and all headers to bump the master branch to version 2.0.0. The name of package might change to ipopo2 to avoid dependency issues.
  3. At this point, you can fork the project and start working on the asyncio conversion. I can also make you a contributor of this project to avoid the fork. We should then use issues on this project to synchronize the parts we update.

Some things to not forget:

  • The code and tests will need a big review to remove the parts of if managing the Python 2+3 usage
  • We can now upgrade the dependencies of ome utility bundles (e.g Paho for the MQTT service)

Note for readers of this issue:
This doesn't mean the v1 will stall, just that their will now be 2 versions of iPOPO until the real death of Python 2 (around June 2024 for Red Hat).
Also, Remote Services and Remote Services Admin can be used to make v1 and v2 instances communicate.

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

Perfect. I will gratefully take this task as a contributor. I will wait for further feedback from you in order to start working on this project. How often do you think we should synchronize ?
If anything I'm available for help.

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

I'll try to do steps 1 and 2 of the previous comment tomorrow (Monday), and I'll add you as contributor.

For the synchronization, I was thinking about something like creating a single issue for the conversion, and adding a comment with the name of the modules and package we intend to work on the next days.

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

Perfect , I agree . Let's stay in touch

from ipopo.

scottslewis avatar scottslewis commented on September 22, 2024

A naive question: What will be the expected user and developer benefits of using asyncio in ipopo?

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

A naive question: What will be the expected user and developer benefits of using asyncio in ipopo?

The benefit will be a boost in scalability and performance because asyncio was written to avoid wasting CPU time on I/O bound tasks or waiting for Network connection. In fact, coroutines and event loops allow developers to execute code only when it's not waiting for I/O or Network connection and to yield control back for other tasks. In phew words the number of threads of iPOPO will be reduced to only a single main thread look at previous comment . If you need further explanation just tell me

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

A naive question: What will be the expected user and developer benefits of using asyncio in ipopo?

Hi Scott,
Like Angelo said, the main advantage of asyncio is to be more efficient handling I/O bound tasks, but also on some waiting APIs (locks, ...).

I would also point out that to gain an overall efficiency using asyncio, the whole framework must be ready to work in asynchronous mode (while keeping the critical sections synchronous, e.g. bundle activation, ...).

Finally, another advantage is the upgrade of the minimal version of Python supported by iPOPO v2 to Python 3.7, which will allow the use of new APIs, to remove compatibility tricks and also to ensure that the module/bundle loading process is up to date (the current one is deprecated in the official API).

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

I've added the v1 (new default branch) and v2 branches to the repository.
@Angeloxx92 is now a contributor of the project.

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

I've added the v1 (new default branch) and v2 branches to the repository.
@Angeloxx92 is now a contributor of the project.

Perfect. Shall we use this issue #108 to synchronize or open a new one?However I'll post Tuesday (tomorrow) the bundle's name on which I'll start to work because I want to organize a good action plan

from ipopo.

tcalmant avatar tcalmant commented on September 22, 2024

from ipopo.

acutaia avatar acutaia commented on September 22, 2024

I've just created the new issue

from ipopo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.