Comments (11)
carstenbauer
commented on May 25, 2024
3
The fact that the main thread is sometimes in :default and sometimes in :interactive, depending on if one starts Julia explicitly with at least one interactive thread, seems to be causing problems (see https://discourse.julialang.org/t/what-is-julia-doing-with-your-threads/110052/20?u=carstenbauer and the related issue #53269).
What can (and should) be done right away is to fix the documentation and document things accurately (I might be able to make a PR tonight).
But I'm still curious what's the reasoning behind this behavior. Why not make the main thread always :interactive?
from julia.
carstenbauer
commented on May 25, 2024
1
That is expected behavior the root task is an interactive task. This is necessary since currently it processes all IO (outside of
@threads).
That's what I thought but
julia -t 3 -E 'Threads.threadpool()'
:default
So it isn't always an interactive task. In fact, assuming that most people don't start Julia with explicit interactive threads, it typically isn't.
from julia.
carstenbauer
commented on May 25, 2024
1
And what the OP forgot to mention, the documentation here is simply wrong:
julia -t 3,1 -E 'Threads.threadpool()'
:interactive # the doc says :default here
This certainly needs to be corrected.
from julia.
carstenbauer
commented on May 25, 2024
1
I can't find a PR for this. I think you might be confusing it with #53117.
Apart from the docs, what about my first point? Seems at least a bit confusing to me that the root thread is in different thread pools depending on how I start Julia.
from julia.
samtkaplan
commented on May 25, 2024
1
@PetrKryslUCSD, I'll try to answer your three questions, but I'm guessing we should move this type of discussion to the discourse thread.
Threads.nthreads()is equivalent toThreads.nthreads(:default), so it only reports the count for the default thread pool. UseThreads.nthreads(:interactive)to see the number of threads in the interactive thread pool.- You are reporting the thread-pools associated with individual threads. You can use
Threads.threadpooltids(:default)andThreads.threadpooltids(:interactive)to see the which thread ids are associated with each pool. Threads.@spawn ...is equivalent toThreads.@spawn :default .... UseThreads.@spawn :interactive ...to run a task on the interactive thread pool.
from julia.
raminammour
commented on May 25, 2024
1
Hey Sam, for reference.
from julia.
vchuravy
commented on May 25, 2024
That is expected behavior the root task is an interactive task. This is necessary since currently it processes all IO (outside of @threads).
from julia.
vtjnash
commented on May 25, 2024
I think someone was working on a docfix for that in a PR recently
from julia.
samtkaplan
commented on May 25, 2024
From my point of view (a simple user), it is confusing that by default code does not use a thread from the default thread pool (assuming that I started Julia with at least one interactive thread). In other words, the statement "by default code does not use a thread from the default thread pool" sounds wrong by virtue of its construction. Not sure if that feed-back can help with the eventual change to the docs, but hope it helps :)
from julia.
PetrKryslUCSD
commented on May 25, 2024
I must be misunderstanding something here. This is what I get when running as a batch and also as interactive:
>>>>>>>>>>>>>>> BATCH <<<<<<<<<<<<<<<<<<<<<<
PS C:\Users\pkonl\Documents\00WIP\FinEtoolsHeatDiff.jl\examples> julia -t 5 .\steady_state\3-d\mwe_tasks.jl
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 3, thread 5, default: Spawned 5.99999e-02
[ Info: Chunk 2, thread 2, default: Spawned 6.39999e-02
[ Info: Chunk 1, thread 3, default: Spawned 4.72000e-01
[ Info: Chunk 4, thread 1, default: Spawned 4.74000e-01
[ Info: 4: Finished 1.77100e+00
[ Info: 3: Finished 1.77200e+00
[ Info: 2: Finished 1.77300e+00
[ Info: 1: Finished 1.77300e+00
[ Info: Finished 1.77400e+00
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 4, thread 1, default: Spawned 0.00000e+00
[ Info: Chunk 3, thread 4, default: Spawned 9.99928e-04
[ Info: Chunk 2, thread 3, default: Spawned 9.99928e-04
[ Info: Chunk 1, thread 2, default: Spawned 9.99928e-04
[ Info: 4: Finished 1.29500e+00
[ Info: 3: Finished 1.29600e+00
[ Info: 1: Finished 1.29700e+00
[ Info: 2: Finished 1.29900e+00
[ Info: Finished 1.29900e+00
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 3, thread 3, default: Spawned 0.00000e+00
[ Info: Chunk 4, thread 1, default: Spawned 0.00000e+00
[ Info: Chunk 2, thread 4, default: Spawned 0.00000e+00
[ Info: Chunk 1, thread 2, default: Spawned 0.00000e+00
[ Info: 1: Finished 1.28900e+00
[ Info: 3: Finished 1.28900e+00
[ Info: 4: Finished 1.29300e+00
[ Info: 2: Finished 1.29600e+00
[ Info: Finished 1.29700e+00
extrema(ts) = (1.29906e+00, 1.30095e+00)
>>>>>>>>>>>>> INTERACTIVE <<<<<<<<<<<<<<<<<<<
PS C:\Users\pkonl\Documents\00WIP\FinEtoolsHeatDiff.jl\examples> julia -t5
julia> include(raw"C:\Users\pkonl\Documents\00WIP\FinEtoolsHeatDiff.jl\examples\steady_state\3-d\mwe_tasks.jl")
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 4, thread 1, default: Spawned 6.00002e-02
[ Info: Chunk 1, thread 3, default: Spawned 4.86000e-01
[ Info: Chunk 2, thread 2, default: Spawned 4.89000e-01
[ Info: Chunk 3, thread 4, default: Spawned 6.40001e-02
[ Info: 4: Finished 1.85500e+00
[ Info: 3: Finished 3.14000e+00
[ Info: 1: Finished 3.14100e+00
[ Info: 2: Finished 3.14400e+00
[ Info: Finished 3.14500e+00
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 1, thread 1, default: Spawned 0.00000e+00
[ Info: Chunk 2, thread 3, default: Spawned 0.00000e+00
[ Info: Chunk 4, thread 5, default: Spawned 0.00000e+00
[ Info: Chunk 3, thread 4, default: Spawned 0.00000e+00
[ Info: 1: Finished 1.30800e+00
[ Info: 4: Finished 1.31200e+00
[ Info: 3: Finished 1.32000e+00
[ Info: 2: Finished 1.32500e+00
[ Info: Finished 1.32600e+00
[ Info: nthreads = 5
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:default, :default, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (5, 0)
[ Info: Chunk 3, thread 4, default: Spawned 0.00000e+00
[ Info: Chunk 4, thread 1, default: Spawned 0.00000e+00
[ Info: Chunk 2, thread 3, default: Spawned 0.00000e+00
[ Info: Chunk 1, thread 2, default: Spawned 0.00000e+00
[ Info: 4: Finished 1.28900e+00
[ Info: 2: Finished 1.28900e+00
[ Info: 1: Finished 1.28900e+00
[ Info: 3: Finished 1.29000e+00
[ Info: Finished 1.29000e+00
extrema(ts) = (1.29278e+00, 1.32864e+00)
(1.29278e+00, 1.32864e+00)
- When running julia interactively, no interactive thread is reported. However,
maxthreadid = 6is printed, when I asked for 5 threads. - When not running interactively, the number of threads and the pool assignations are identical.
Now I start julia with two thread pools configured explicitly:
>>>>>>>>>>>>> INTERACTIVE <<<<<<<<<<<<<<<<<<<
$ julia -t4,1
julia> include(raw"C:\Users\pkonl\Documents\00WIP\FinEtoolsHeatDiff.jl\examples\steady_state\3-d\mwe_tasks.jl")
[ Info: nthreads = 4
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:interactive, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (4, 1)
[ Info: Chunk 1, thread 3, default: Spawned 5.90000e-02
[ Info: Chunk 4, thread 2, default: Spawned 5.15000e-01
[ Info: Chunk 3, thread 5, default: Spawned 5.19000e-01
[ Info: Chunk 2, thread 4, default: Spawned 6.30000e-02
[ Info: 1: Finished 1.79800e+00
[ Info: 4: Finished 1.80000e+00
[ Info: 2: Finished 1.80200e+00
[ Info: 3: Finished 1.80600e+00
[ Info: Finished 1.80700e+00
[ Info: nthreads = 4
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:interactive, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (4, 1)
[ Info: Chunk 2, thread 2, default: Spawned 0.00000e+00
[ Info: Chunk 3, thread 4, default: Spawned 9.99928e-04
[ Info: Chunk 1, thread 5, default: Spawned 0.00000e+00
[ Info: Chunk 4, thread 3, default: Spawned 9.99928e-04
[ Info: 3: Finished 1.28700e+00
[ Info: 4: Finished 1.28700e+00
[ Info: 1: Finished 1.28800e+00
[ Info: 2: Finished 1.28900e+00
[ Info: Finished 1.28900e+00
[ Info: nthreads = 4
[ Info: maxthreadid = 6
[ Info: nthreadpools = 2
[ Info: threadpool.(1:Threads.nthreads()) = [:interactive, :default, :default, :default]
[ Info: Threads.threadpoolsize.((:default, :interactive)) = (4, 1)
[ Info: Chunk 4, thread 5, default: Spawned 0.00000e+00
[ Info: Chunk 1, thread 2, default: Spawned 0.00000e+00
[ Info: Chunk 3, thread 4, default: Spawned 0.00000e+00
[ Info: Chunk 2, thread 3, default: Spawned 0.00000e+00
[ Info: 3: Finished 1.28400e+00
[ Info: 2: Finished 1.28700e+00
[ Info: 4: Finished 1.28800e+00
[ Info: 1: Finished 1.28900e+00
[ Info: Finished 1.28900e+00
extrema(ts) = (1.29101e+00, 1.29140e+00)
(1.29101e+00, 1.29140e+00)
- The number of threads is reported as 4. Is it because only the default pool threads are counted?
- When listing the thread pool assignations, one of them is reported as interactive. Why?
- The tasks are however all started from the default pool.
from julia.
carstenbauer
commented on May 25, 2024
Doc fix: #53388
Regarding the question of why the main thread can be in either the default or interactive threadpool, I'll open a separate issue to discuss this, because the present issue is a bit convoluted. (done: #53389)
from julia.
Related Issues (20)
- Broken linkerscript, julia fails to link with modern lld versions HOT 2
- Documentation bug: `MainInclude`
- Confusing error when defining a method within nested testsets
- [FEATURE] Using Right Arrow Key to Auto Complete Virtual Text?
- should matrix-vector mul! also protect against aliasing? HOT 2
- code reports an error to line 0 HOT 6
- Average of Dates HOT 9
- safety consideration for operations with incompletely initialized types HOT 2
- Dimension check too restrictive in copytrito! and lacpy!
- LLVMPtr's address space annotation is lost with opaque pointers
- Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffad71a9977 -- gc_setmark_pool_ HOT 8
- confusing error thrown by `display` HOT 1
- Make `ScopedValues.get` a method of `Base.get` HOT 6
- Julia without libstdc++ HOT 2
- `--threads` option seems to now result in segfault in `jl_init_thread_heap` (`julia/src/gc.c:3992`) HOT 1
- `strings/annotated` test relies on that StyledStrings have been loaded
- Not all `StridedArray`s follow the strided array API
- GC could do with more consolidated intro user documentation
- Stackoverflow without stacktrace when `convert`ing large tuple HOT 4
- `atexit()` finalizers should not run after encountering `SIGSEGV`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from julia.