Comments (5)
Jishaku intentionally doesn't redirect stdout or stderr automatically because it is not async safe - these interfaces were designed "as is" close to 40 years ago with no inherent consideration of the complexity that we get out of modern systems. To have any hope of regulating it "safely" - I would have to take over the entire control flow of your script.
Since that is unreasonable within the design specifications of Jishaku, here are a number of reasons why I will not/can not capture stdout or stderr in good conscience:
Unix streams have no concept of 'context'
When text is written to stdout, the only information that comes with it is the text itself. There are no categories, labels, or denominators under which a program can definitively determine what a given text buffer was for.
This therefore makes it impossible for a capturer to distinguish "text intended for capture" and "unrelated other stdout content". Consider for example that during your REPL session, your logger pushes a critical error related to your bot's functionality to stderr. Not only will this wash up as unintended noise in your REPL session - but as redirection is non-forwarding, it will prevent this error from turning up in your actual logs - giving you no record that it happened.
There is only one stdout/stderr
Due to the simplicity of basic Unix streams as described above - there can be only one stdout in a program, and one handler of it.
This primarily has significance in non-forwarding capture due to its potential to make a capture meaningless - even if I redirect_stdout
, if something else does it immediately after me, then just like how I am intercepting text from reaching the real stdout, the new acquirer will stop any text from reaching my capturer, and I will receive nothing.
Asynchronous environments do not guarantee reverse disengagement of context managers
Global-bound synchronous context managers are often built on the assumption that context managers will be exited in the reverse order that they are entered, due to the limitations of the with
syntax and therefore the assumed flow in a fully synchronous context.
Let's first consider the simplest situation:
with A() as a:
...
In this code, the context manager flow looks like this:
>> A (enter)
<< A (exit)
This flow is very simple, and there's very little that can go wrong here.
Now, let's consider a more complicated structure:
with A() as a:
with B() as b:
...
with C() as c:
with D() as d:
...
For this code, the context manager flow looks like this:
>> A (enter)
>> B (enter)
<< B (exit)
>> C (enter)
>> D (enter)
<< D (exit)
<< C (exit)
<< A (exit)
From this, we can begin to notice a pattern. Whenever any context manager is exited, it is always the last-entered non-exited context manager, or more broadly:
In a fully synchronous environment, for any given context manager, that context manager cannot exit until all contained context managers have exited first.
Now, let's discuss the behavior of redirect_stdout
. In plain English, redirect_stdout
:
- Stores the current value of
sys.stdout
for later restoration - Overwrites
sys.stdout
with a string buffer to serve as the fake stdout for the duration of the context manager. - Overwrites
sys.stdout
once the context manager exits with the version stored at the beginning.
We can see how this system preserves a sensible stack, even in a nested case:
with redirect_stdout(...) as A:
with redirect_stdout(...) as B:
...
...
Even if we were to create a complicated nest of context managers as before, due to context manager exit always occuring in the reverse order of enter, we will always end like we started.
That is, if we are in a synchronous context.
When we bring asyncio into the equation, things get complicated. Coroutines can suspend and resume during execution, and other coroutines can operate in the interim. While context manager order is guaranteed on a local scale, it is no longer guaranteed on a global scale.
Let's now consider a pair of coroutines that look like this:
# These coroutines are functionally the same, but have different symbol names for illustrative purposes.
async def foo():
with redirect_stdout(...) as A:
await asyncio.sleep(5)
...
async def bar():
with redirect_stdout(...) as B:
await asyncio.sleep(5)
...
Thanks to asyncio, we can run both of these coroutines at the same time. Let's give that a try.
>> A (enter)
>> B (enter)
<< A (exit)
<< B (exit)
As we can see, when we break the exit guarantee, things stop working as intended. In this case, even though both coroutines have now returned, sys.stdout
is left as a dead text buffer. From now on, not only will nothing make it to the regular stdout, but the remaining text buffer will sit there as a memory sink, slowly increasing memory usage every time anything writes to stdout.
While this seems like a manufactured scenario, such a thing could easily happen by accident if one runs multiple REPLs at the same time, as suspension is allowed every time a coroutine awaits.
As the maintainer of a package I wish to preserve as "production-safe", it's pretty clear why I can't let things like this happen automatically and silently just through basic use.
If you really want to redirect stdout, if such a specific case where it is necessary arises, implicit imports allow you to do so in one block:
with io!.StringIO() as f, contextlib!.redirect_stdout(f):
print("hi")
await _author.send(f.getvalue()) # send stdout contents to author
from jishaku.
Just use _ctx.author.send
in your event
from jishaku.
Uhh that's not what I mean.
Basically, let's say I have something in a cog that runs on the on_message() event.
If that prints to stdout or stderr, I want to receive the output in DMs.
_ctx.author.send
is for jsk py
as far as I am aware.
from jishaku.
Oh anything, including outside of jsk py? Any reason you can't check the logs on your VPS for that?
from jishaku.
The way I manage my processes unfortunately splits off stdout and stderr, meaning it's not really feasible for me to check VPS logs and sometimes I don't notice an exception because it silently appears because I don't watch my VPS logs all the time.
from jishaku.
Related Issues (20)
- jsk ast fails with an error in certain cases HOT 2
- ability to add user who can run jishaku on the fly HOT 2
- Button disable
- Add Whois command HOT 2
- Unsupport discord.py v1.7.x HOT 1
- Python 3.9.13 not supported HOT 1
- Proper user configurable jsk py dict vars HOT 1
- change jishaku commands names HOT 1
- Jishaku version occasionally appears as vNone in selftest HOT 4
- Adding asqlite to options for the sql features HOT 3
- Unhandled code branch in `jishaku voice` command.
- Cog already loaded when loading it HOT 1
- I canβt use jishaku shell command HOT 2
- `jishaku timeit` raises an AttributeError upon use. HOT 1
- Tracking: need for a new release
- KeyError on Python version 3.11.0 HOT 3
- AttributeError: 'Context' object has no attribute 'add_reaction' HOT 2
- Question about next possible release to pypi HOT 2
- Convert message links to Message objects in REPLs HOT 1
- [Tracking] import_expression doesn't function correctly in 3.12 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jishaku.