Giter VIP home page Giter VIP logo

pyo3-log's Introduction

pyo3-log

Actions Status codecov docs

A bridge to send Rust's log messages over to Python. Meant to help logging from pyo3 native extensions.

Read the documentation before using.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

pyo3-log's People

Contributors

a1phyr avatar annapst avatar congyuwang avatar danielvschoor avatar davidhewitt avatar dependabot[bot] avatar diddlum avatar edmorley avatar g-bauer avatar oriontvv avatar sandhose avatar v02460 avatar vorner avatar yodaldevoid avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

pyo3-log's Issues

`pyo3_init` resulting in a segementation fault

Hi,

I am trying to add pyo3-log in my code but it is resulting in a seg fault.

Here is the rust code

mod executors;
mod io_helpers;
mod request_handler;
mod routers;
mod server;
mod shared_socket;
mod types;
mod web_socket_connection;
// use pyo3_log::{Caching, Logger};

use server::Server;
use shared_socket::SocketHeld;

// pyO3 module
use pyo3::prelude::*;

#[pymodule]
pub fn robyn(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    // the pymodule class to make the rustPyFunctions available
    pyo3_log::init();

    m.add_class::<Server>()?;
    m.add_class::<SocketHeld>()?;
    pyo3::prepare_freethreaded_python();

    Ok(())
}

and here is the python code

# default imports
import os
import asyncio
from inspect import signature
import multiprocessing as mp
from robyn.events import Events

# custom imports and exports
from .robyn import SocketHeld
from .argument_parser import ArgumentParser
from .responses import static_file, jsonify
from .dev_event_handler import EventHandler
from .processpool import spawn_process
from .log_colors import Colors
from .ws import WS
from .router import Router, MiddlewareRouter, WebSocketRouter

# 3rd party imports and exports
from multiprocess import Process
from watchdog.observers import Observer

mp.allow_connection_pickling()

import logging

FORMAT = '%(levelname)s %(name)s %(asctime)-15s %(filename)s:%(lineno)d %(message)s'
logging.basicConfig(format=FORMAT)
logging.getLogger().setLevel(logging.INFO)
logging.info("Test 1")


class Robyn:
    """This is the python wrapper for the Robyn binaries."""

    def start(self, url="127.0.0.1", port=5000):
        """
        Starts the server

        :param port int: reperesents the port number at which the server is listening
        """

        if not self.dev:
            processes = []
            workers = self.workers
            socket = SocketHeld(url, port)
            for _ in range(self.processes):
                copied_socket = socket.try_clone()
                p = Process(
                    target=spawn_process,
                    args=(
                        self.directories,
                        self.headers,
                        self.router.get_routes(),
                        self.middleware_router.get_routes(),
                        self.web_socket_router.get_routes(),
                        self.event_handlers,
                        copied_socket,
                        workers,
                    ),
                )
                p.start()
                processes.append(p)

            print("Press Ctrl + C to stop \n")
            try:
                for process in processes:
                    process.join()
            except KeyboardInterrupt:
                print(f"\n{Colors.BOLD}{Colors.OKGREEN} Terminating server!! {Colors.ENDC}")
                for process in processes:
                    process.kill()
        else:
            event_handler = EventHandler(self.file_path)
            event_handler.start_server_first_time()
            print(
                f"{Colors.OKBLUE}Dev server initialised with the directory_path : {self.directory_path}{Colors.ENDC}"
            )
            observer = Observer()
            observer.schedule(event_handler, path=self.directory_path, recursive=True)
            observer.start()
            try:
                while True:
                    pass
            finally:
                observer.stop()
                observer.join()

I was able to identify the problem to this snippet by using comments and print statements.

mp.allow_connection_pickling()


def spawn_process(
    directories, headers, routes, middlewares, web_sockets, event_handlers, socket, workers
):
    """
    This function is called by the main process handler to create a server runtime.
    This functions allows one runtime per process.

    :param directories tuple: the list of all the directories and related data in a tuple
    :param headers tuple: All the global headers in a tuple
    :param routes Tuple[Route]: The routes touple, containing the description about every route.
    :param middlewares Tuple[Route]: The middleware router touple, containing the description about every route.
    :param web_sockets list: This is a list of all the web socket routes
    :param event_handlers Dict: This is an event dict that contains the event handlers
    :param socket SocketHeld: This is the main tcp socket, which is being shared across multiple processes.
    :param process_name string: This is the name given to the process to identify the process
    :param workers number: This is the name given to the process to identify the process
    """

    # platform_name = platform.machine()
    if sys.platform.startswith("win32") or sys.platform.startswith("linux-cross"):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
    else:
        # uv loop doesn't support windows or arm machines at the moment
        # but uv loop is much faster than native asyncio
        import uvloop

        uvloop.install()
        loop = uvloop.new_event_loop()
        asyncio.set_event_loop(loop)

    server = Server()

    for directory in directories:
        route, directory_path, index_file, show_files_listing = directory
        server.add_directory(route, directory_path, index_file, show_files_listing)

    for key, val in headers:
        server.add_header(key, val)

    for route in routes:
        route_type, endpoint, handler, is_async, number_of_params = route
        server.add_route(route_type, endpoint, handler, is_async, number_of_params)

    for route in middlewares:
        route_type, endpoint, handler, is_async, number_of_params = route
        server.add_middleware_route(route_type, endpoint, handler, is_async, number_of_params)

    if "startup" in event_handlers:
        server.add_startup_handler(event_handlers[Events.STARTUP][0], event_handlers[Events.STARTUP][1])

    if "shutdown" in event_handlers:
        server.add_shutdown_handler(event_handlers[Events.SHUTDOWN][0], event_handlers[Events.SHUTDOWN][1])

    for endpoint in web_sockets:
        web_socket = web_sockets[endpoint]
        print(web_socket.methods)
        server.add_web_socket_route(
            endpoint,
            web_socket.methods["connect"],
            web_socket.methods["close"],
            web_socket.methods["message"],
        )

    try:
        server.start(socket, workers)
        loop = asyncio.get_event_loop()
        loop.run_forever()
    except KeyboardInterrupt:
        loop.close()

I am getting the following error code:

๐Ÿ“ฆ Built wheel for CPython 3.10 to /var/folders/4k/tfv95dys49s5syt_t24swgvh0000gn/T/.tmpslmZ2C/robyn-0.16.3-cp310-cp310-macosx_10_7_x86_64.whl
๐Ÿ›   Installed robyn-0.16.3
[1]    65284 segmentation fault  python3 integration_tests/base_routes.py

and I am using maturin as the development system.

Do you happen to know a fix for this?

Logging from pure Rust

Is it possible to log from pure Rust(not wrapped with pyfunction or pymethods)?

Background:
I'm starting a Rust websocket server from Python, which then runs in its own thread. Logging does not work as soon as I move into pure Rust code

I know there's a slim chance, but I figured I'd ask anyway

TypeError: not all arguments are converted during string formatting

Hello! I am using a custom logginghandler for my Discord bot. This bot isusing lavasnek_rs for music playing in a server. When lavasnek_rs tries to log their messages i get this error:
image

My logginghandler is as follows:
image

What am I doing wrong and what is the possible solution?

Document interaction with starting other threads in Rust

The following (minimal) example does not work.

use log::debug;
use pyo3::prelude::*;
use pyo3_log;
use rayon::prelude::*;

#[pymodule]
fn my_module(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
    pyo3_log::init();

    #[pyfn(m)]
    #[pyo3(name = "test")]
    fn py_test<'py>(_py: Python<'py>, n_procs: u64) {
        (0..n_procs).into_par_iter().for_each(|i| debug!("{}", i));
    }
    Ok(())
}

The problem concerns the logging macros inside parallel iterators of rayon crate.

PyO3 0.15 support.

Presumably, there is no need of big change except for updating to version number to v0.5.0 and update pyo3 dependency to v0.15.

For now using pyo3-log gives me two version of pyo3 mixed if I use v0.15 in my project, which results in python failure: "Fatal Python error: PyThreadState_Get: no current thread"

Migrate to pyo3 0.13

First thank you for the crate, works like a charm!

Do you plan to upgrade it to pyo3 0.13? Not very important but it would clear my dependency trees for projects which have migrated.

Not compatible with pyo 0.21.1

At least according to the dependency resolver

cargo add pyo3-log
    Updating crates.io index
      Adding pyo3-log v0.9.0 to dependencies.
    Updating crates.io index
error: failed to select a version for `pyo3-ffi`.
    ... required by package `pyo3 v0.16.2`
    ... which satisfies dependency `pyo3 = ">=0.15, <0.21"` of package `pyo3-log v0.9.0`
    ... which satisfies dependency `pyo3-log = "^0.9.0"` of package `REDACTED v0.1.0 (REDACTED)`
versions that meet the requirements `=0.16.2` are: 0.16.2

the package `pyo3-ffi` links to the native library `python`, but it conflicts with a previous package which links to `python` as well:
package `pyo3-ffi v0.21.1`
    ... which satisfies dependency `pyo3-ffi = "=0.21.1"` of package `pyo3 v0.21.1`
    ... which satisfies dependency `pyo3 = "^0.21.1"` of package `REDACTED v0.1.0 (REDACTED)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the `links = "python"` value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

failed to select a version for `pyo3-ffi` which could resolve this conflict

`log_enabled!` macro behavior

Hello!
I've encountered some strange behavior regarding log_enabled! macro that seems like a bug.
If it called before any call of another level-logging macros (except 'trace!'), it does not set the check properly: it always returns true no matter what logging settings are enabled.
Here is a patch with modified example that produces this issue.
It looks like there is no such a problem in flexi_logger, for example.

diff --git a/examples/hello_world/hw.py b/examples/hello_world/hw.py
index c45c67a..f10136b 100755
--- a/examples/hello_world/hw.py
+++ b/examples/hello_world/hw.py
@@ -5,6 +5,6 @@ import hello_world
 
 FORMAT = '%(levelname)s %(name)s %(asctime)-15s %(filename)s:%(lineno)d %(message)s'
 logging.basicConfig(format=FORMAT)
-logging.getLogger().setLevel(logging.INFO)
+logging.getLogger().setLevel(logging.CRITICAL)
 logging.info("Test 1")
 hello_world.log_hello()
diff --git a/examples/hello_world/src/lib.rs b/examples/hello_world/src/lib.rs
index bec87c7..11c31b7 100644
--- a/examples/hello_world/src/lib.rs
+++ b/examples/hello_world/src/lib.rs
@@ -1,12 +1,15 @@
-use log::{debug, trace, info};
+use log::{debug, trace, info, log_enabled, Level};
 use pyo3::prelude::*;
 use pyo3::wrap_pyfunction;
 use pyo3_log::{Caching, Logger};
 
 #[pyfunction]
 fn log_hello() {
+    println!("First call {}", log_enabled!(Level::Info));
     trace!("xyz");
+    println!("Second call {}", log_enabled!(Level::Info));
     debug!("stuff2");
+    println!("Third call {}", log_enabled!(Level::Info));
     debug!("Stuff");
     info!("Hello {}", "world");
     info!("Hello 2{}", "world");

Change logging level during runtime

First of all, thanks for making this crate. It's been very useful!

I was hoping to be able to disable the logger from Python by using something like logging.getLogger("mymodulename").set_log_level(51), but that only disables calls from Python (e.g. logging.getLogger("mymodulename").warning("...") gets disabled) and doesn't disable actual logging from Rust. I see there is documentation related to manually clearing the cache, and I thought a Python-exposed function that clears the cache might be the best solution. I can't seem to figure out how to write that though, any help? The conflict seems to be coming from a handler being created in my lib.rs and not knowing how to use that handler in a pyo3 function later on. I'm new to Rust so please bear with me.

Emitting a `log` message during interpreter finalization will cause a panic

In my project, I have a Rust struct that will print a log message when it is dropped. In my Python code, this Rust struct lives in global scope and is not deallocated by the interpreter until the Python process exits.

When the Python process exits and drops my Rust struct, pyo3_log::Logger panics when trying to log the message. This is because it tries to acquire the GIL, but since the interpreter is in the process of finalization, the GIL is not available, so pyo3::Python::with_gil panics

Due to PyO3/pyo3#2102, this panic manifests as a SIGABRT being sent to the thread.

--

I think one potential solution is to check that the interpreter is in an initialized state before trying to acquire the GIL using pyo3::ffi::Py_IsInitialized -- however the function is unsafe and I'm not familiar enough with Python internals to know when it is safe to call.

logging from a submodule of a namespaced package

First off, thanks for making this crate, it's exactly what I needed. I may have made my project structure too ambitious, though, and I'm trying to get everything to work out.

I'm writing a namespace package: let's call the project space and it installs as name.space. I want to write part of it in Rust for speed, and so I have a mixed project with a Cargo lib space that I renamed as _space (so I import name.space._space to access the Rust extension).

There's a CLI w/ Click defined in name.space.__main__.py and an entrypoint in pyproject.toml, and that tool can use the Rust extension functions once the package is installed. All of this works suspiciously great so far!

What's tripping me up right now is logging from Rust. I'd like my (Python) CLI to initialize and format a logger for name.space, but it seems like it won't display logs from Rust. I can still set up a logger in the REPL and use the extension module: it logs with the name space, rather than name.space. This might be the issue, but I'm not sure at this point.

Tag 0.8.1

Doesn't look like v0.8.1 got tagged.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.