Comments (3)
MichaelHatherly
commented on June 24, 2024
2
@goerz had some of the logging extensions in https://github.com/JuliaLogging/LoggingExtras.jl not covered your needs for that kind of thing? Yes, they won't capture the output of print/println, but if you're in control of the source of the logging then maybe just using a logger is the appropriate way to do this. IOCapture was always meant to be a very small self-contained module to do just the one thing.
from iocapture.jl.
goerz
commented on June 24, 2024
2
These optimizations also use logging at all levels, which is separate from the information printed to stdout: I want the Info prefix in the logging output, but not in every line of the printed output. Moreover, the printed output is potentially generated by user code, since it's an extensible framework where people can pass their own optimizer.
I sympathize with wanting to keep IOCapture small. Quite honestly, if I was maintaining IOCapture, I'd probably pass on adding the FirstLastBuffer (but I just wanted to put it out there).
On the other hand, adding a hidden or official option to replace the internal IOBuffer is pretty minimal. To be specific, I opened alternative PRs #22 and #23. It would be great if you'd consider either one of these. Otherwise, I'd have to copy over the entire capture function into my code. That works for me, too, but it feels like a shame to have the duplication.
(And I absolutely acknowledge I'm abusing IOCapture for something it wasn't really intended for)
from iocapture.jl.
goerz
commented on June 24, 2024
Oh, and just to clarify what kind of output the take! for FirstLastBuffer produces, with an example:
Using FirstLastBuffer(1024) (keep the first and last kB) in the context of an example from QuantumControl, an optimization that would normally capture the output
iter. J_T ∫gₐ(t)dt J ΔJ_T ΔJ secs
0 1.06e-01 0.00e+00 1.06e-01 n/a n/a 2.2
1 7.44e-02 1.58e-02 9.01e-02 -3.15e-02 -1.57e-02 4.6
2 7.37e-02 3.52e-04 7.40e-02 -7.03e-04 -3.52e-04 3.4
3 7.30e-02 3.33e-04 7.33e-02 -6.65e-04 -3.32e-04 3.4
4 7.24e-02 3.15e-04 7.27e-02 -6.29e-04 -3.14e-04 3.4
5 7.18e-02 2.99e-04 7.21e-02 -5.97e-04 -2.98e-04 3.4
6 7.12e-02 2.84e-04 7.15e-02 -5.68e-04 -2.84e-04 3.4
7 7.07e-02 2.71e-04 7.09e-02 -5.41e-04 -2.71e-04 3.4
8 7.02e-02 2.59e-04 7.04e-02 -5.17e-04 -2.58e-04 3.3
9 6.97e-02 2.48e-04 6.99e-02 -4.95e-04 -2.48e-04 3.3
10 6.92e-02 2.38e-04 6.94e-02 -4.76e-04 -2.38e-04 3.3
11 6.87e-02 2.29e-04 6.90e-02 -4.59e-04 -2.29e-04 3.3
12 6.83e-02 2.22e-04 6.85e-02 -4.44e-04 -2.22e-04 3.5
13 6.79e-02 2.17e-04 6.81e-02 -4.33e-04 -2.16e-04 3.3
14 6.74e-02 2.13e-04 6.76e-02 -4.26e-04 -2.13e-04 3.4
15 6.70e-02 2.11e-04 6.72e-02 -4.23e-04 -2.11e-04 3.3
16 6.66e-02 2.13e-04 6.68e-02 -4.25e-04 -2.13e-04 3.3
17 6.61e-02 2.18e-04 6.64e-02 -4.35e-04 -2.18e-04 3.5
18 6.57e-02 2.28e-04 6.59e-02 -4.55e-04 -2.27e-04 3.4
19 6.52e-02 2.44e-04 6.54e-02 -4.89e-04 -2.44e-04 3.4
20 6.47e-02 2.71e-04 6.49e-02 -5.42e-04 -2.71e-04 3.4
21 6.40e-02 3.13e-04 6.43e-02 -6.26e-04 -3.13e-04 3.4
22 6.33e-02 3.79e-04 6.37e-02 -7.57e-04 -3.78e-04 3.4
23 6.23e-02 4.82e-04 6.28e-02 -9.63e-04 -4.81e-04 3.4
24 6.10e-02 6.47e-04 6.17e-02 -1.29e-03 -6.46e-04 3.4
25 5.92e-02 9.10e-04 6.01e-02 -1.82e-03 -9.09e-04 3.4
26 5.66e-02 1.30e-03 5.79e-02 -2.61e-03 -1.30e-03 3.5
27 5.30e-02 1.79e-03 5.48e-02 -3.58e-03 -1.79e-03 3.4
28 4.87e-02 2.14e-03 5.09e-02 -4.28e-03 -2.14e-03 3.5
29 4.47e-02 2.04e-03 4.67e-02 -4.08e-03 -2.04e-03 3.6
30 4.16e-02 1.54e-03 4.31e-02 -3.07e-03 -1.53e-03 3.4
31 3.96e-02 1.02e-03 4.06e-02 -2.02e-03 -1.01e-03 3.5
32 3.82e-02 6.66e-04 3.89e-02 -1.33e-03 -6.61e-04 3.4
33 3.73e-02 4.68e-04 3.78e-02 -9.33e-04 -4.65e-04 3.4
34 3.66e-02 3.57e-04 3.69e-02 -7.10e-04 -3.54e-04 3.3
35 3.60e-02 2.89e-04 3.63e-02 -5.77e-04 -2.87e-04 3.5
36 3.55e-02 2.47e-04 3.58e-02 -4.92e-04 -2.45e-04 3.3
37 3.51e-02 2.20e-04 3.53e-02 -4.37e-04 -2.18e-04 3.4
38 3.47e-02 2.02e-04 3.49e-02 -4.02e-04 -2.00e-04 3.6
39 3.43e-02 1.91e-04 3.45e-02 -3.81e-04 -1.89e-04 3.4
40 3.39e-02 1.85e-04 3.41e-02 -3.68e-04 -1.83e-04 3.4
41 3.36e-02 1.82e-04 3.37e-02 -3.62e-04 -1.80e-04 3.4
42 3.32e-02 1.80e-04 3.34e-02 -3.59e-04 -1.79e-04 3.4
43 3.28e-02 1.80e-04 3.30e-02 -3.58e-04 -1.78e-04 3.3
44 3.25e-02 1.80e-04 3.27e-02 -3.59e-04 -1.78e-04 3.4
45 3.21e-02 1.81e-04 3.23e-02 -3.60e-04 -1.79e-04 3.6
46 3.18e-02 1.82e-04 3.19e-02 -3.62e-04 -1.80e-04 3.6
47 3.14e-02 1.83e-04 3.16e-02 -3.64e-04 -1.81e-04 3.5
48 3.10e-02 1.84e-04 3.12e-02 -3.65e-04 -1.82e-04 3.5
49 3.07e-02 1.85e-04 3.09e-02 -3.68e-04 -1.83e-04 3.4
50 3.03e-02 1.86e-04 3.05e-02 -3.70e-04 -1.84e-04 3.3
now captures the output
iter. J_T ∫gₐ(t)dt J ΔJ_T ΔJ secs
0 1.06e-01 0.00e+00 1.06e-01 n/a n/a 2.2
1 7.44e-02 1.58e-02 9.01e-02 -3.15e-02 -1.57e-02 4.6
2 7.37e-02 3.52e-04 7.40e-02 -7.03e-04 -3.52e-04 3.4
3 7.30e-02 3.33e-04 7.33e-02 -6.65e-04 -3.32e-04 3.4
4 7.24e-02 3.15e-04 7.27e-02 -6.29e-04 -3.14e-04 3.4
5 7.18e-02 2.99e-04 7.21e-02 -5.97e-04 -2.98e-04 3.4
6 7.12e-02 2.84e-04 7.15e-02 -5.68e-04 -2.84e-04 3.4
7 7.07e-02 2.71e-04 7.09e-02 -5.41e-04 -2.71e-04 3.4
8 7.02e-02 2.59e-04 7.04e-02 -5.17e-04 -2.58e-04 3.3
9 6.97e-02 2.48e-04 6.99e-02 -4.95e-04 -2.48e-04 3.3
10 6.92e-02 2.38e-04 6.94e-02 -4.76e-04 -2.38e-04 3.3
11 6.87e-02 2.29e-04 6.90e-02 -4.59e-04 -2.29e-04 3.3
12 6.83e-02 2.22e-04 6.85e-02 -4.44e-04 -2.22e-04 3.5
13 6.79e-02 2.17e-04 6.81e-0…
…04 3.58e-02 -4.92e-04 -2.45e-04 3.3
37 3.51e-02 2.20e-04 3.53e-02 -4.37e-04 -2.18e-04 3.4
38 3.47e-02 2.02e-04 3.49e-02 -4.02e-04 -2.00e-04 3.6
39 3.43e-02 1.91e-04 3.45e-02 -3.81e-04 -1.89e-04 3.4
40 3.39e-02 1.85e-04 3.41e-02 -3.68e-04 -1.83e-04 3.4
41 3.36e-02 1.82e-04 3.37e-02 -3.62e-04 -1.80e-04 3.4
42 3.32e-02 1.80e-04 3.34e-02 -3.59e-04 -1.79e-04 3.4
43 3.28e-02 1.80e-04 3.30e-02 -3.58e-04 -1.78e-04 3.3
44 3.25e-02 1.80e-04 3.27e-02 -3.59e-04 -1.78e-04 3.4
45 3.21e-02 1.81e-04 3.23e-02 -3.60e-04 -1.79e-04 3.6
46 3.18e-02 1.82e-04 3.19e-02 -3.62e-04 -1.80e-04 3.6
47 3.14e-02 1.83e-04 3.16e-02 -3.64e-04 -1.81e-04 3.5
48 3.10e-02 1.84e-04 3.12e-02 -3.65e-04 -1.82e-04 3.5
49 3.07e-02 1.85e-04 3.09e-02 -3.68e-04 -1.83e-04 3.4
50 3.03e-02 1.86e-04 3.05e-02 -3.70e-04 -1.84e-04 3.3
In reality, I wouldn't have stopped the optimization after 50 iterations, but it would keep going for many thousands iterations (even millions, although not for problems that take 3.5 seconds per iteration).
I'd probably post-process that captured output a little bit more and remove the two partial lines (replacing them with a single … line). But even without that, I'm pretty happy with the "truncated" captured output, and it's something I'd archive in a JLD file together with the optimization result.
from iocapture.jl.
Related Issues (10)
- iocapture -> capture, don't export
- Rename throwerrors -> rethrow HOT 1
- TagBot trigger issue HOT 6
- Set rethrow to false HOT 1
- Capturing output in multithreaded context HOT 1
- Fails to capture long output from C stdio HOT 3
- Hang when captureing output from MathLink HOT 7
- Add a `pass_through` option HOT 6
- Declare as "stable" HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iocapture.jl.