Giter VIP home page Giter VIP logo

Comments (16)

adefossez avatar adefossez commented on August 18, 2024 1

Hello @zeddan, sadly Github doesn't provide machines with macos 12 for the automated build of the DiffQ package. The same is true for lameenc.

It should be possible to build those packages locally but I'm not sure how to do it with Docker.

from docker-facebook-demucs.

adefossez avatar adefossez commented on August 18, 2024 1

@zeddan thanks to this issues from Mac M1 users (facebookresearch/demucs#246) I realized I had forgotten to upload source distributions for diffq, only precompiled once. Could you give it one more go now ? I think it should work as long as a compiler is available in the docker image.
Thanks :)

from docker-facebook-demucs.

Reidsy avatar Reidsy commented on August 18, 2024 1

In the meantime, as a workaround, I threw together a dockerfile that compiles lameenc while building the docker image.

https://gist.github.com/Reidsy/07db8e4d484d61f84408cab98fc24965

from docker-facebook-demucs.

radostyle avatar radostyle commented on August 18, 2024 1 
demucs on  main [?] ➜ make run track=myfile.mp3
[+] Building 1.8s (9/10)
 => [internal] load build definition from Dockerfile                                                          0.0s
 => => transferring dockerfile: 1.19kB                                                                        0.0s
 => [internal] load .dockerignore                                                                             0.0s
 => => transferring context: 2B                                                                               0.0s
 => [internal] load metadata for docker.io/nvidia/cuda:11.8.0-base-ubuntu22.04                                0.5s
 => [1/7] FROM docker.io/nvidia/cuda:11.8.0-base-ubuntu22.04@sha256:3a85383782012c87cd78f9b47c33b6105b2b6e17  0.0s
 => CACHED [4/7] WORKDIR /lib/demucs                                                                          0.0s
 => CACHED [5/7] RUN python3 -m pip install -e . --no-cache-dir                                               0.0s
 => ERROR [6/7] RUN python3 -m demucs -d cpu test.mp3                                                         1.2s
------
 > [6/7] RUN python3 -m demucs -d cpu test.mp3:
#9 0.989 Traceback (most recent call last):
#9 0.989   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
#9 0.989     return _run_code(code, main_globals, None,
#9 0.989   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
#9 0.989     exec(code, run_globals)
#9 0.989   File "/usr/lib/demucs/demucs/__main__.py", line 7, in <module>
#9 0.989     from .separate import main
#9 0.989   File "/usr/lib/demucs/demucs/separate.py", line 14, in <module>
#9 0.989     import torchaudio as ta
#9 0.989   File "/usr/local/lib/python3.10/dist-packages/torchaudio/__init__.py", line 1, in <module>
#9 0.989     from torchaudio import (  # noqa: F401
#9 0.989   File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension.py", line 135, in <module>
#9 0.989     _init_extension()
#9 0.989   File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension.py", line 105, in _init_extension
#9 0.989     _load_lib("libtorchaudio")
#9 0.989   File "/usr/local/lib/python3.10/dist-packages/torchaudio/_extension.py", line 52, in _load_lib
#9 0.989     torch.ops.load_library(path)
#9 0.989   File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 643, in load_library
#9 0.990     ctypes.CDLL(path)
#9 0.990   File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
#9 0.990     self._handle = _dlopen(self._name, mode)
#9 0.990 OSError: /usr/local/lib/python3.10/dist-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZNK3c
104impl13OperatorEntry24assertSignatureIsCorrectENS0_12CppSignatureEb
------
executor failed running [/bin/sh -c python3 -m demucs -d cpu test.mp3]: exit code: 1
make: *** [build] Error 1

Fails like this on my Apple M1 Pro OS 12.4
Latest commit a776015

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Hello @zeddan , thank you so much for reporting this issue. As @adefossez commented there's no way right now to test the build step with MacOS 12 with GitHub Actions. It's true that they'll support the new version soon but now it's on beta (see: github/roadmap#240).

I'll try to install MacOS 12 on my local machine and I've created this issue #15 to add a Github Action to test the execution of the Makefile on different platforms. It's in progress but I hope I'll add a pull request soon.

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Hi @adefossez @zeddan , I've just upgraded my laptop to last version MacOS 12 and the process worked as expected. I've got only this error: killed due to the Docker Desktop virtual needed more RAM resources to execute the whole process. Now I've set 8gb of RAM and 2Gb for Swap.

I've created a repository in Docker Hub to store the docker image there instead of building it every time in the local environment. I hope that this change will improve the usage of the Makefile and reduce the issues related to the building process. Checkout https://hub.docker.com/r/xserrat/facebook-demucs/tags.

I'm going to create a pull request to update the docker image used in the Makefile. Feel free to suggest any improvement! :)

Thanks so much!!

from docker-facebook-demucs.

zeddan avatar zeddan commented on August 18, 2024

Hi @adefossez @xserrat

Thanks for looking into this issue!

I re-cloned the repository and tried again, and this time it succeeds on the diffq package but still seems to yield an error on the lameenc step:

docker build -t facebook/demucs:latest .
[+] Building 8.5s (10/12)
 => [internal] load build definition from Dockerfile                                                           0.0s
 => => transferring dockerfile: 906B                                                                           0.0s
 => [internal] load .dockerignore                                                                              0.0s
 => => transferring context: 2B                                                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.8                                                  1.9s
 => [auth] library/python:pull token for registry-1.docker.io                                                  0.0s
 => [1/8] FROM docker.io/library/python:3.8@sha256:881e0df149c29af8b29a973a9e80814dae6cddf123fe38a0bcac71864c  0.0s
 => CACHED [2/8] RUN apt install git                                                                           0.0s
 => CACHED [3/8] RUN mkdir -p /lib/demucs                                                                      0.0s
 => CACHED [4/8] WORKDIR /lib/demucs                                                                           0.0s
 => CACHED [5/8] RUN git clone -b main --single-branch https://github.com/facebookresearch/demucs /lib/demucs  0.0s
 => ERROR [6/8] RUN python3 -m pip install -e .                                                                6.5s
------
 > [6/8] RUN python3 -m pip install -e .:
#10 0.806 Obtaining file:///lib/demucs
#10 1.320 Collecting dora-search
#10 1.748   Downloading dora_search-0.1.7.tar.gz (68 kB)
#10 1.955   Installing build dependencies: started
#10 5.541   Installing build dependencies: finished with status 'done'
#10 5.543   Getting requirements to build wheel: started
#10 5.637   Getting requirements to build wheel: finished with status 'done'
#10 5.639     Preparing wheel metadata: started
#10 5.744     Preparing wheel metadata: finished with status 'done'
#10 5.848 Collecting diffq>=0.2.1
#10 5.919   Downloading diffq-0.2.2.tar.gz (44 kB)
#10 6.131 Collecting julius>=0.2.3
#10 6.220   Downloading julius-0.2.6.tar.gz (58 kB)
#10 6.463 ERROR: Could not find a version that satisfies the requirement lameenc>=1.2 (from demucs) (from versions: none)
#10 6.463 ERROR: No matching distribution found for lameenc>=1.2
#10 6.467 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
#10 6.467 You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c python3 -m pip install -e .]: exit code: 1
make: *** [build] Error 1

@xserrat you don't happen to know something else I can try, since you have got it working? I'm running on a fresh install of MacOS 12 so it's possible that I'm missing some configuration.

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Hi @zeddan ,

Could you try to run the following command to build the image from scratch without cache:

docker build --no-cache -t facebook/demucs:latest .

Then, execute again the make run track=xxxx.

If the error persists, I've just merged this PR #19 related to the issue #17 to prevent possible errors when building the image. You can update the main branch and execute the make run track=xxxx again and it should work.

Let me know how it was!

from docker-facebook-demucs.

adefossez avatar adefossez commented on August 18, 2024

I think the author of lameenc and forgot to include the source distribution for lameenc. I'll open a bug upstream.

from docker-facebook-demucs.

adefossez avatar adefossez commented on August 18, 2024

An issue already exist: chrisstaite/lameenc#13

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

I didn't realise it was related to the new Mac M1 😄 🙏🏽 . Perfect @adefossez 🚀

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Hi @zeddan,

Could you try to build the image again with the latest changes that support ARM64 as well as other improvements (#24)?

Just to make sure we can close this issue.

Thanks!

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

@radostyle thanks for reporting this issue. It seems to be related with an incompatibility with the version of torchaudio that demucs is using. I'll investigate in more detail 🙏🏼

from docker-facebook-demucs.

radostyle avatar radostyle commented on August 18, 2024

I'm excited to try it out. Let me know if there is something to test.

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Hi @radostyle , sorry for my delay replying about this issue.

I've just merged the fix #37 that should solve the issue.

Could you try it again using the version v1.0.0 of this repository?

Thanks in advance.

from docker-facebook-demucs.

xserrat avatar xserrat commented on August 18, 2024

Closing issue after some time waiting for response. I assume it worked.

from docker-facebook-demucs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.