Giter VIP home page Giter VIP logo

Comments (11)

CYYJL avatar CYYJL commented on July 18, 2024 2

Hi, I found that the issue with the initialization of wandb was due to its inability to retrieve the GPU model and CPU model during initialization. However, when I commented out that part of the code, wandb was able to initialize normally. It might be because I am using the latest version of the Nvidia driver, version 555.99, and wandb cannot get the corresponding GPU model from the driver, causing the error.
d541ccf5671eb0d2f690a5a12dc2a77

from wandb.

CYYJL avatar CYYJL commented on July 18, 2024 1

@Wissotsky @zzk2021
Hi,My solution is to avoid using pynvml to get GPU information. The downside is that you won't be able to see GPU and CPU-related information on the wandb website. In the code, you can comment out the code I showed in the probe function of the library file wandb/sdk/internal/system/assets, and then wandb will work normally.
image

from wandb.

benbowler avatar benbowler commented on July 18, 2024

I'm seeing this today after upgrading to the v555 of the NVIDIA drivers. I can confirm commenting out this block bypasses the error for now.

from wandb.

zzk2021 avatar zzk2021 commented on July 18, 2024

image
same problem, how to fix it?

from wandb.

Wissotsky avatar Wissotsky commented on July 18, 2024

Can concur the same problem occurring with driver ver 555.99

from wandb.

fmamberti-wandb avatar fmamberti-wandb commented on July 18, 2024

Thank you for surfacing this. I will raise with our engineering team and we will keep you posted with the progress

from wandb.

exalate-issue-sync avatar exalate-issue-sync commented on July 18, 2024

WandB Internal User commented:
zzk2021 commented:
image
same problem, how to fix it?

from wandb.

exalate-issue-sync avatar exalate-issue-sync commented on July 18, 2024

WandB Internal User commented:
Wissotsky commented:
Can concur the same problem occurring with driver ver 555.99

from wandb.

exalate-issue-sync avatar exalate-issue-sync commented on July 18, 2024

WandB Internal User commented:
CYYJL commented:
@Wissotsky @zzk2021
Hi,My solution is to avoid using pynvml to get GPU information. The downside is that you won't be able to see GPU and CPU-related information on the wandb website. In the code, you can comment out the code I showed in the probe function of the library file wandb/sdk/internal/system/assets, and then wandb will work normally.
image

from wandb.

cyber-chris avatar cyber-chris commented on July 18, 2024

Thanks, looks like the fix worked.

In case anyone else is having this issue, make sure you upgrade to 0.17.2

from wandb.

dmitryduev avatar dmitryduev commented on July 18, 2024

Please upgrade to version 0.17.2 and turn on our new backend wandb-core with wandb.require("core") after importing wandb. No source code modification should be necessary.

from wandb.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.