Does it build the docker image on every make run track=mysong.mp3 now or only when necessary (remote facebook/demucs branch is ahead of local branch), i.e. does it auto-check if building is necessary?
Building on every run seems not optimal to me either, because here at least it takes a good while to build (just did a make build as per your advice above before I saw your new post).

Originally posted by @nofishonfriday in #25 (reply in thread)

The build is ok, but when running with make run path/to/file.mp3 I get the following error:

docker run --rm -i \
		--name=demucs \
		-v /Users/loretoparisi/Documents/Projects/AI/docker-facebook-demucs-main/input:/data/input \
		-v /Users/loretoparisi/Documents/Projects/AI/docker-facebook-demucs-main/output:/data/output \
		-v /Users/loretoparisi/Documents/Projects/AI/docker-facebook-demucs-main/models:/data/models \
		facebook/demucs:latest \
		"python3 -m demucs.separate --dl -n demucs -d cpu --out /data/output --models /data/models \
		/data/input/"
Traceback (most recent call last):
  File "/opt/conda/envs/demucs/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/envs/demucs/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/lib/demucs/demucs/separate.py", line 13, in <module>
    import torchaudio as ta
  File "/opt/conda/envs/demucs/lib/python3.9/site-packages/torchaudio/__init__.py", line 15, in <module>
    from torchaudio.backend import (
  File "/opt/conda/envs/demucs/lib/python3.9/site-packages/torchaudio/backend/__init__.py", line 2, in <module>
    from . import utils
  File "/opt/conda/envs/demucs/lib/python3.9/site-packages/torchaudio/backend/utils.py", line 7, in <module>
    from . import (
  File "/opt/conda/envs/demucs/lib/python3.9/site-packages/torchaudio/backend/soundfile_backend.py", line 15, in <module>
    import soundfile
  File "/opt/conda/envs/demucs/lib/python3.9/site-packages/soundfile.py", line 142, in <module>
    raise OSError('sndfile library not found')
OSError: sndfile library not found
make: *** [run] Error 1

I assume that libsndfile should be in the Dockerfile right?

Also, after make build I cannot see the downloaded model's files in the volumes folders:

ip-192-168-178-30:docker-facebook-demucs-main loretoparisi$ tree -L 3
.
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── input
├── models
└── output

In order to improve the experience when reporting a bug, we could create a bug report template so that bug issues will follow the same format.

I was wondering what would need to be done to make this version able to utilize the nvidia gpu? I am guessing would need to start Dockerfile using a nvidia image, and then install python, and everything else? Then would just need to add option in Dockerfile to download the gpu model?

In order to understand how the Docker image can be used, we should add a README.md with some information about it.

As the title says. This requires a small change in the Dockerfile.

Thank you for your work.
Would it be possible to add the shifts option to your script?

In order to reduce the possible issues during the building process of the image, we could create the docker image in Docker Hub to make the usage of the Makefile easier.

Contact Details

[email protected]

What happened?

I am trying to run the amazing demucs-software on my Apple M1 Pro using the docker image but getting the following error.

In which operating system you found the issue?

MacOS

Relevant log output

Selected model is a bag of 4 models. You will see that many progress bars per track.
Separated tracks will be stored in /tmp/output/mdx_extra_q
Separating track /tmp/d93bdecf9dd25d29394238327db07b0fabfdeaba860ec3bd3e2c341d3d0ea0ca.mp3

  0%|                                                                                   | 0.0/33.0 [00:00<?, ?seconds/s][W NNPACK.cpp:79] Could not initialize NNPACK! Reason: Unsupported hardware.

Contact Details

[email protected]

What happened?

After cloning the repo, both trying "make build" or "make run track=test01.mp3" gives the same AssertionError

In which operating system you found the issue?

Linux

Relevant log output

Successfully built julius dora-search antlr4-python3-runtime treetable
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 165, in exc_logging_wrapper
    status = run_func(*args)
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 389, in run
    to_install = resolver.get_installation_order(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 188, in get_installation_order
    weights = get_topological_weights(
  File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 276, in get_topological_weights
    assert len(weights) == expected_node_count
AssertionError
The command '/bin/sh -c python3 -m pip install -e . --no-cache-dir' returned a non-zero code: 2
make: *** [Makefile:60: build] Error 2

Contact Details

[email protected]

What happened?

trying to build a docker container with the dockerfile from this repository, receiving this error. this happens with both make build and make run commands. Changing the image to a aarm64 base does not alleviate the error.

test.mp3 is a file in the appropriate location in input folder.

would really appreciate any guidance here. thanks for putting this together by the way. will be really cool once its up and running.

In which operating system you found the issue?

MacOS

Relevant log output

=> ERROR [6/7] RUN python3 -m demucs -d cpu test.mp3                                                  3.3s
------                                                                                                      
 > [6/7] RUN python3 -m demucs -d cpu test.mp3:                                                             
0.831 Downloading: "https://dl.fbaipublicfiles.com/demucs/hybrid_transformer/955717e8-8726e21a.th" to /data/models/hub/checkpoints/955717e8-8726e21a.th
100%|██████████| 80.2M/80.2M [00:01<00:00, 48.1MB/s]
3.255 Selected model is a bag of 1 models. You will see that many progress bars per track.
3.255 Separated tracks will be stored in /usr/lib/demucs/separated/htdemucs
3.255 Separating track test.mp3
  0%|                                                                                   | 0.0/23.4 [00:00<?, ?seconds/s]Segmentation fault
------
Dockerfile:27
--------------------
  25 |     RUN python3 -m pip install -e . --no-cache-dir
  26 |     # Run once to ensure demucs works and trigger the default model download
  27 | >>> RUN python3 -m demucs -d cpu test.mp3 
  28 |     # Cleanup output - we just used this to download the model
  29 |     RUN rm -r separated
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 -m demucs -d cpu test.mp3" did not complete successfully: exit code: 139
make: *** [build] Error 1

In order to test that the execution of the Demucs library is correct with the Dockerfile and Makefile configs, we could create a Github workflow to be executed on a pull request and also on the main branch.

The workflow should:

• Execute the make build
• Execute the make run with a sample audio file
• Run on ubuntu, macos-10, macos-11 and windows.

I notice this version is using an obsolete github project.

The maintained demucs has moved to
https://github.com/adefossez/demucs

At the moment, running the image opens up a shell, in which the user has to input the python command to actually separate the tracks. This makes it fairly annoying when trying to set up a pipeline that can directly split the track based on input.

Check out the screenshot below for reference:

However, when I try to mirror the make run command to pass in the python command to run demucs directly, that doesn't really work as it simply exists without executing the command in the first place:

docker run --rm --name=demucs \
    -v /path/to/input:/data/input \
    -v /path/to/output:/data/output \
    -v /path/to/models:/data/models \
    xserrat/facebook-demucs:latest \
    /bin/bash -c "python3 -m demucs -n htdemucs --out /data/output /data/input/01-kaligiyuNTEgadA_galgunu-kIravANi_seg_30.mp3"

It would be nice to build an image that actually does this for us, hiding away the details of executing demucs under the hood with python. I built a wrapper image in my own pipeline that allows me to do this, which we can adapt. In this way, it would be easier for users to simply pass in environment variables as necessary and run the image in a single fell swoop.

docker run --rm --name=demucs \
    -e GPU=false
    -e MP3OUTPUT=true
    -e MODEL=htdemucs
    -v /path/to/input:/data/input \
    -v /path/to/output:/data/output \
    -v /path/to/models:/data/models \ 
    wrapped-demucs \
    01-kaligiyuNTEgadA_galgunu-kIravANi_seg_30.mp3

In this case, wrapped-demucs refers to the wrapper image that I built on top of the existing xserrat/facebook-demucs image.

Contact Details

[email protected]

What happened?

Hi!

Thanks for taking the time to work on dockerizing demucs 🙌

OS: macOS Monterey 12.0
Docker: 4.2.0

When trying to run a track separation the build fails on installation, please see the Relevant log output section.

However, if RUN python3 -m pip install -e . is replaced with manual installations like the following:

RUN python3 -m pip install dora-search
RUN python3 -m pip install diffq>=0.2.1
RUN python3 -m pip install flake8
RUN python3 -m pip install hydra-colorlog>=1.1
...

the installation of diffq completes. Then the problem instead becomes installing lameenc, with the following error message:

docker build -t facebook/demucs:latest .
[+] Building 2.4s (16/27)
 => [internal] load build definition from Dockerfile                           0.0s
 => => transferring dockerfile: 1.16kB                                         0.0s
 => [internal] load .dockerignore                                              0.0s
 => => transferring context: 2B                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.8                  1.6s
 => [auth] library/python:pull token for registry-1.docker.io                  0.0s
 => [ 1/23] FROM docker.io/library/python:3.8@sha256:68bddbf6e88c9c88d3238e13  0.0s
 => CACHED [ 2/23] RUN apt install git                                         0.0s
 => CACHED [ 3/23] RUN mkdir -p /lib/demucs                                    0.0s
 => CACHED [ 4/23] WORKDIR /lib/demucs                                         0.0s
 => CACHED [ 5/23] RUN git clone -b main --single-branch https://github.com/f  0.0s
 => CACHED [ 6/23] RUN python3 -m pip install dora-search                      0.0s
 => CACHED [ 7/23] RUN python3 -m pip install diffq>=0.2.1                     0.0s
 => CACHED [ 8/23] RUN python3 -m pip install flake8                           0.0s
 => CACHED [ 9/23] RUN python3 -m pip install hydra-colorlog>=1.1              0.0s
 => CACHED [10/23] RUN python3 -m pip install hydra-core>=1.1                  0.0s
 => CACHED [11/23] RUN python3 -m pip install julius>=0.2.3                    0.0s
 => ERROR [12/23] RUN python3 -m pip install lameenc>=1.2                      0.7s
------
 > [12/23] RUN python3 -m pip install lameenc>=1.2:
#16 0.670 ERROR: Could not find a version that satisfies the requirement lameenc (from versions: none)
#16 0.670 ERROR: No matching distribution found for lameenc
#16 0.672 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
#16 0.672 You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c python3 -m pip install lameenc>=1.2]: exit code: 1
make: *** [build] Error 1

What I've tried:

• Upgrading pip with RUN python3 -m pip install --upgrade pip
• Using a later python version by switching to FROM python:3.9 at the top

Thanks in advance!

In which operating system you found the issue?

MacOS

Relevant log output

[+] Building 5.1s (9/11)
 => [internal] load build definition from Dockerfile                           0.0s
 => => transferring dockerfile: 541B                                           0.0s
 => [internal] load .dockerignore                                              0.0s
 => => transferring context: 2B                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.8                  0.8s
 => [1/8] FROM docker.io/library/python:3.8@sha256:68bddbf6e88c9c88d3238e13f0  0.0s
 => CACHED [2/8] RUN apt install git                                           0.0s
 => CACHED [3/8] RUN mkdir -p /lib/demucs                                      0.0s
 => CACHED [4/8] WORKDIR /lib/demucs                                           0.0s
 => CACHED [5/8] RUN git clone -b main --single-branch https://github.com/fac  0.0s
 => ERROR [6/8] RUN python3 -m pip install -e .                                4.2s
------
 > [6/8] RUN python3 -m pip install -e .:
#9 0.864 Obtaining file:///lib/demucs
#9 1.246 Collecting dora-search
#9 1.481   Downloading dora_search-0.1.7.tar.gz (68 kB)
#9 1.654   Installing build dependencies: started
#9 3.874   Installing build dependencies: finished with status 'done'
#9 3.877   Getting requirements to build wheel: started
#9 3.975   Getting requirements to build wheel: finished with status 'done'
#9 3.977     Preparing wheel metadata: started
#9 4.080     Preparing wheel metadata: finished with status 'done'
#9 4.132 ERROR: Could not find a version that satisfies the requirement diffq>=0.2.1 (from demucs) (from versions: 0.0.1, 0.1.0, 0.1.1, 0.2.0)
#9 4.132 ERROR: No matching distribution found for diffq>=0.2.1
#9 4.136 WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
#9 4.136 You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c python3 -m pip install -e .]: exit code: 1
make: *** [build] Error 1

Contact Details

[email protected]

What happened?

Hello, I have a query about the interplay between processing duration and memory utilization. When working with a WAV audio file that's an hour long, my server's Python 3 script hits a peak memory usage of 22.3GB. In an attempt to address this, I've incorporated flags like -e PYTORCH_NO_CUDA_MEMORY_CACHING=1 and --segment 2, yet the memory footprint stubbornly stays the same regardless of whether I set segment to 1, 2, or 3. The exact command sequence I'm employing is:

Bash：
_docker run --rm -i --name=demucs1 --gpus all -e PYTORCH_NO_CUDA_MEMORY_CACHING=1
-v /data/demucs/input:/data/input
-v /data/demucs/output:/data/output
-v /data/demucs/models:/data/models
xserrat/facebook-demucs:latest
"python3 -m demucs -n htdemucs_ft --out /data/output --mp3 --two-stems vocals --segment 2 /data/input/test_60m1.wav"

Do you have any suggestions or strategies to alleviate this substantial memory demand during processing?

In which operating system you found the issue?

Linux

Relevant log output

No response

Hello @xserrat , on the 12th of November I will release a new version of Demucs. This should only require minor command line changes :) I'll submit a PR soon but wanted to give you a heads up.

Hi @xserrat,
I just updated demucs including the CLI flags. You should remove the --dl flag, which is no longer supported (it is always implied). Also you could replace the default model used (-n flag) from demucs to demucs_quantized which will be much smaller and equivalent quality.

I can send a PR tomorrow :)

Enforce the version v4.0.0 to make execution stable before introducing new improvements from the Demucs lib.

Working on:

• MacOS 12 (Intel)
• MacOS 14 (Chip M2)
• Ubuntu 22.04

Due to the Docker image allocated in DockerHub is not updated automatically based on the Dockerfile changes (waiting for #27), we need a fallback to build the image manually without running make build (discussed here).