Huge virtual memory usage on Windows about julia HOT 5 CLOSED

"Virtual memory used by Julia"

Where did this data come from?
Does this refer to the "Virtual Size" in Process Explorer?

I have 32GB of physical memory.
Windows seems to allocate 2~3TB of virtual memory space for Edge.
Maybe Edge has a much bigger problem than julia?

Maybe related: Unreasonably huge process virtual memory size reported by Process Explorer

from julia.

In general, virtual memory isn't really meaningful (at least on 64 bit systems). You have roughly infinite address space, available, so as long as you don't bring the memory into resident memory, it's all fairly imaginary.

from julia.

"Virtual memory used by Julia"

Where did this data come from? Does this refer to the "Virtual Size" in Process Explorer?

I have 32GB of physical memory. Windows seems to allocate 2~3TB of virtual memory space for Edge. Maybe Edge has a much bigger problem than julia?

Maybe related: Unreasonably huge process virtual memory size reported by Process Explorer

No, it's actually the Private Bytes in Process Explorer, which is about 300MB in the screenshot of Edge. In addition, my workstation was initially configured to have total virtual memory of about 32GB, running a few instances of Julia in parallel will result in errors like "Cannot load xxx.dll due to insufficient virtual memory" (I cannot remember the exact message, but it's something like that). After raising the maximum limit of virtual memory to 1TiB, now I can open about 64 instances in parallel without actually doing any calculation.
So I think in my case, Julia seems to use too much virtual memory as soon as it starts up, and uses up the virtual memory at a very fast rate.

In general, virtual memory isn't really meaningful (at least on 64 bit systems). You have roughly infinite address space, available, so as long as you don't bring the memory into resident memory, it's all fairly imaginary.

Within a specific amount (like tens or hundreds of GB), I do agree that virtual memory is not a problem. However, for some unknown reason, on my workstation, the amount of virtual memory is extremely large and prevents other programs (including other instances of Julia) from running.

By the way, is there any way to know how the virtual memory is being used, so that I might be able to compare the usage on both systems, and see what might be wrong?

from julia.

What if you set the environment variable OPENBLAS_NUM_THREADS=1 before starting julia? Does that change anything?

from julia.

What if you set the environment variable OPENBLAS_NUM_THREADS=1 before starting julia? Does that change anything?

It works! The amount of used virtual memory has decreased to about 180MB.

After trying some different values of OPENBLAS_NUM_THREADS, it seems that even setting it to 256 (number of logical cores in the Workstation) cannot replicate the usage of 17GB virtual memory, but I think that might be another issue for openBLAS, instead of Julia.

Anyway, the problem has been solved, thanks to all you people for helping me with this!

from julia.