Comments (14)
robbert-harms
commented on September 3, 2024
hi Mark,
Thank you for the compliments. Hopefully we can fix this for you.
The message PLATFORM_NOT_FOUND_KHR is indeed a typical OpenCL related error. Could you try running the command line utility "clinfo" and see if that works and finds any OpenCL devices?
My guess is that the nvidia/amd drivers were updated and that the OpenCL drivers are now either missing or incorrectly linked to the graphic driver. Probably clinfo will crash as well and then it's a job for the IT department.
Let me know how it goes,
Best,
Robbert
from mdt.
mwagshul
commented on September 3, 2024
Thanks Robbert, unfortunately my system doesn't recognize that command so is this a deeper problem? Again, odd because I know the package worked for a long time. Thanks again for the help with getting this running.
Best,
Mark
from mdt.
robbert-harms
commented on September 3, 2024
Not at all. clinfo is a standalone utility which you probably haven't installed yet. Probably you need something like "yum install clinfo".
from mdt.
mwagshul
commented on September 3, 2024
Hmm, that would require admin privileges, which would take a few days for
them to get around to. Is there a version which I can install locally (e.g.
pip install)?
…On Wed, Jun 16, 2021 at 8:39 AM Robbert Harms ***@***.***> wrote:
Not at all. clinfo is a standalone utility which you probably haven't
installed yet. Probably you need something like "yum install clinfo".
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#39 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEJXEVLXHZULVNFFJEM4OA3TTCLQJANCNFSM46XXFH5Q>
.
from mdt.
robbert-harms
commented on September 3, 2024
Unfortunately not. You could try opening a Python shell and typing:
import pyopencl as cl
cl.get_platforms()it will probably crash because the CL drivers are not initialized correctly. Having an independent tool (clinfo) verify this would add some extra information.
from mdt.
mwagshul
commented on September 3, 2024
But no surprise here, get the same error as before (I think this is the
function that you're code was failing on):
pyopencl._cl.LogicError: clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR
…On Wed, Jun 16, 2021 at 9:47 AM Robbert Harms ***@***.***> wrote:
Unfortunately not. You could try opening a Python shell and typing:
import pyopencl as clcl.get_platforms()
it will probably crash because the CL drivers are not initialized
correctly. Having an independent tool (clinfo) verify this would add some
extra information.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#39 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEJXEVKGGKD3NGPWNIPFYHDTTCTOJANCNFSM46XXFH5Q>
.
from mdt.
robbert-harms
commented on September 3, 2024
Yes, but the point is that it is not an MDT or MOT problem (my packages), but a problem with OpenCL. I am aware that that doesn't solve your issue, but it makes it clear that it technically not an MDT or MOT issue.
My advice would be to ask your IT department if they can verify your OpenCL installation.
from mdt.
mwagshul
commented on September 3, 2024
Yes, thanks for the help. I'm in touch with them and they're working on it.
Best,
Mark
…On Wed, Jun 16, 2021 at 10:44 AM Robbert Harms ***@***.***> wrote:
Yes, but the point is that it is not an MDT or MOT problem (my packages),
but a problem with OpenCL. I am aware that that doesn't solve your issue,
but it makes it clear that it technically not an MDT or MOT issue.
My advice would be to ask your IT department if they can verify your
OpenCL installation.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#39 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEJXEVKGQSEHGEORPIB6TJLTTC2FXANCNFSM46XXFH5Q>
.
from mdt.
robbert-harms
commented on September 3, 2024
Did it work? If so, then we can close this issue. :)
from mdt.
mwagshul
commented on September 3, 2024
No, sorry. Been busy with other things. Very strange behavior - my system
admin can run all of the programs just fine, even when he logs into my
account. But I can't run anything, and neither can a few other users who
I've checked with.
The only thing I can think of, which I've asked him to check, is that he
can directly ssh into the GPU node. I can't do that. I need to qrsh in,
which I have had some problems with in other applications (Parallel
processing in Matlab specifically). But still not sure why this would make
a difference.
Thanks for asking, and if you have any ideas, let me know.
Best,
Mark
…On Tue, Jun 29, 2021 at 5:34 AM Robbert Harms ***@***.***> wrote:
Did it work? If so, then we can close this issue. :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#39 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEJXEVOWNFQMDABXAFE6BELTVGHRZANCNFSM46XXFH5Q>
.
from mdt.
mwagshul
commented on September 3, 2024
Robbert,
I seem to have "fixed" the problem, but the processing seems to be very,
very slow. Instead of using the globally installed version of MDT, I
installed it locally and using that version, it at least doesn't fail right
away as it did before when using the globally installed MDT version. But
now it seems to be stuck on the "We will use the optimizer Powell with
default settings." step for a long time (my past experience was that it
only took a few minutes to complete (dataset is 246 volumes). I am running
this on a 8-processor NVIDIA GPU.
Thoughts? Is there a way to run the processing in "verbose" mode to see
what it's doing?
Thanks,
Mark
…On Thu, Jul 1, 2021 at 7:18 PM Mark Wagshul ***@***.***> wrote:
No, sorry. Been busy with other things. Very strange behavior - my system
admin can run all of the programs just fine, even when he logs into my
account. But I can't run anything, and neither can a few other users who
I've checked with.
The only thing I can think of, which I've asked him to check, is that he
can directly ssh into the GPU node. I can't do that. I need to qrsh in,
which I have had some problems with in other applications (Parallel
processing in Matlab specifically). But still not sure why this would make
a difference.
Thanks for asking, and if you have any ideas, let me know.
Best,
Mark
On Tue, Jun 29, 2021 at 5:34 AM Robbert Harms ***@***.***>
wrote:
> Did it work? If so, then we can close this issue. :)
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#39 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEJXEVOWNFQMDABXAFE6BELTVGHRZANCNFSM46XXFH5Q>
> .
>
from mdt.
robbert-harms
commented on September 3, 2024
Hi Mark,
Happy to hear it works, sad to hear it is slow. Can you provide me with the rest of the log output? Could you additionally run mdt-list-devices and tell me the output? I have the idea that it is using a different device for the computations and not your nvidia GPU.
Unfortunately MDT does not have an additional verbose mode. The log file is what I use for debugging purposes.
Best,
Robbert
from mdt.
mwagshul
commented on September 3, 2024
Well, I think I've solved the problem. It seems to have to do with the
memory request when I qrsh into the GPU node. If I don't request a
particular block of memory, I get those errors relating to OpenCL, but when
I request 30G (it seems to use up to ~ 20G peak), then it runs fine in ~ 5
minutes. Mystery solved, although the errors we were getting were odd and
not obviously related to the memory requested. It's also odd that this was
working a few months ago without the memory request.
Thanks for all your help, I think we're back in business!
Mark
…On Tue, Jul 20, 2021 at 10:12 AM Robbert Harms ***@***.***> wrote:
Hi Mark,
Happy to hear it works, sad to hear it is slow. Can you provide me with
the rest of the log output? Could you additionally run mdt-list-devices and
tell me the output? I have the idea that it is using a different device for
the computations and not your nvidia GPU.
Unfortunately MDT does not have an additional verbose mode. The log file
is what I use for debugging purposes.
Best,
Robbert
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#39 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEJXEVKQHTQPQ67P5HLH5CDTYV75RANCNFSM46XXFH5Q>
.
from mdt.
robbert-harms
commented on September 3, 2024
Hi Mark, happy that it works! Good luck with your science.
Best, Robbert
from mdt.
Related Issues (20)
- How to compose a new composite model HOT 1
- Noise estimation HOT 8
- Singularity runs, but gives semi-random results HOT 8
- Initialization HOT 2
- Value Error cannot find model 'NODDI' HOT 3
- AttributeError: module 'pyopencl.cltypes' has no attribute '4' HOT 5
- fixing parameters in fit process
- Numpy errors HOT 1
- Issue with invalid array size HOT 2
- mdt-list-models is empty, unable to select expected models for fit
- T1 fit
- RuntimeError: clEnqueueNDRangeKernel failed: OUT_OF_RESOURCES in fitting with CHARMED_r2
- pyopencl.cffi_cl.RuntimeError: clBuildProgram failed: BUILD_PROGRAM_FAILURE - HOT 4
- Running test file HOT 2
- Installation on Centos7
- Running MDT via Docker HOT 1
- `get_data` and `np.bool`, possibly others, trigger deprecation errors from NiBabel as of nibabel>=5.0.0 and numpy>=1.24.0 HOT 1
- Stuck at "We will use the optimizer Powell with default settings." HOT 2
- Documentation links HOT 1
- issues in NODDI maps using MDT fitting HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mdt.