Using local machine and restrict the maximum CPU can be used about flux-core HOT 7 CLOSED

I gave a test for my use case, seems all works fine! Thanks a lot for the help, I close the issue as resolved.

from flux-core.

For multi-user, you may be able to edit the systemd unit file to set up resource control e.g. AllowedCPUs and MemoryMax, thus restricting flux to a given set of cores and memory using cgroups.

The admin guide describes how to set up multi-user Flux which is a little more involved than what @grondo is describing, so his suggestion may be a better starting point.

from flux-core.

There is no real downside except that you will be oversubscribing resources.

from flux-core.

Flux will detect the resources available to it at startup (using hwloc), so an easy way to run flux on a subset of resources is to launch flux start under taskset(1) or hwloc-bind(1).

grondo@pi3:~$ flux start flux resource list -s free -o rlist
     STATE NNODES   NCORES    NGPUS LIST
      free      1        8        0 rank0/core[0-7]
grondo@pi3:~$ hwloc-bind core:0-3 flux start flux resource list -s free -o rlist
     STATE NNODES   NCORES    NGPUS LIST
      free      1        4        0 rank0/core[0-3]

Even if you specified cores 4-7 here, Flux will renumber the cores to 0-3, however when launching job tasks they should still be constrained to real cores 4-7 since Flux will index the cores relative to those core it has access.

If you want to run a multi-user instance of Flux, then that would be a slightly different setup.

from flux-core.

Cool, thanks a lot for the help! I think that what I need, the single user solution is fit for my use case for the moment.

I notice with --test-size <num> can create more brokers than CPUs, is that possible let's say using 2 CPUs in total but can further separate it to run 4 jobs with each one using 0.5 CPUs? (Sorry if I confused with the terminology)

from flux-core.

Running multiple brokers on the same node will oversubscribe resources. This is mainly used for testing, but some users find this practice useful when performance is not a concern, e.g. for CI jobs. Flux uses this in its own testsuite to simulate a multi-node instance on a single host.

To specifically answer your question:

let's say using 2 CPUs in total but can further separate it to run 4 jobs with each one using 0.5 CPUs? (Sorry if I confused with the terminology)

Yes, if you run 2 brokers on 2 cores, your Flux "instance" will see 4 cores and may schedule up to two 'tasks' per core.

from flux-core.

So I need to use --test-size=2 I assume, it says "a test instance" is there any downside of using it, for my case to use local machine as a cluster I think it cover the use case?

from flux-core.