Comments (3)
Should we convert this issue to a discussion?
from flux-core.
Thank you for doing this again @vsoch and @milroy!
emoji jobids
This could be part of a larger discussion on Flux jobids and multiple representations depending on use context:
(bring on the flux job id
command)
- jobids are 64 bit numbers called FLUIDs. Like UUIDs but are only unique within a given Flux instance. (
flux job id --to=dec
) - hexadecimal for slightly more compact representation (
flux job id --to=hex
) - F58 (base58) default encoding for most compact representation without easily mistaken characters
- dotted hexadecimal for use as a KVS key (
flux job id --to=dothex
andflux job id --to=kvs
) - a sequence of words for speaking over the phone, e.g. to a hotline staff member (
flux job id --to=words
) - and of course emoji encoding (base576) (
flux job id --to=emoji
)
flux filename
I assume you mean flux filemap
? That is a good idea for a tutorial topic. I don't have a good idea of an example for the tutorial though. Maybe run a batch job that downloads a parallel program source to /tmp
, builds it locally, maps the build dir, then uses the shell stage-in
plugin to run the built parallel program without need of a network filesystem?
flux exec
This doesn't hurt to have in a tutorial. Noting that this utility can be used with a single-user instance (batch/alloc jobs) is probably worth doing.
Other ideas:
-
Description of Flux URIs, the
flux uri
utility, and supported "TARGETs" forflux uri
and other commands likeflux proxy TARGET
,flux top TARGET
,flux shutdown TARGET
-
I can't remember if we already have a short tutorial for
flux bulksubmit
. I was thinking it would be good to adapt one of the GNU parallel tutorials usingflux bulksubmit
. You could demonstrate how bulksubmit could be used to parallelize and monitor a more traditional set of tasks (non HPC workload) underflux start
on a single machine. Then we could show how the same set of tasks can be scaled up by using a multinode instance. Running the Flux testsuite under Flux is a good demonstration, though long. -
Demonstrate the
flux batch
/flux alloc
--conf
option, e.g. run an instance with a node exclusive match policy and exclude rank 0 viaflux batch --conf=sched-fluxion-resource.match-policy=firstnodex --conf=resource.exclude=0 OPTIONS.. SCRIPT
-
Demonstrate the
--add-file=
option for submission commands, perhaps in combination with multiline batch directives. -
Another interesting tutorial would be to demonstrate the resiliency of Flux batch jobs. Launch an instance with 4 brokers. Terminate one broker and show that the instance is still running with one "node" marked down.
from flux-core.
flux filename
Yeah that was a typo - translation error between brain and fingers ;) The cases I always use filemap for are when I have multiple physical nodes and need to send over a file to them. E.g., to start usernetes I have a join-command (secret) that is generated by the control-plane and then gets moved over.
I can't remember if we already have a short tutorial for flux bulksubmit
That's a good suggestion (I don't remember) but I don't know how to use it that well, so we should if we don't.
These suggestions are great - thanks @grondo !
from flux-core.
Related Issues (20)
- valgrind reports new leaks with hwloc 2.10
- broker crash in `content_cache_destroy` HOT 2
- `job-ingest: fluid_init failed` when launching 16 brokers per node on 1040 nodes HOT 7
- discussion: go bindings for flux-core HOT 3
- Cray MPI launches as singletons on frontier HOT 10
- shell: support dumping hwloc XML to `HWLOC_XMLFILE`
- Need a bulk submission tool for `flux batch`
- shell taskmap `block` scheme ignores its arguments
- flux-job: support `MPIR_executable_path` and `MPIR_server_arguments` in attach HOT 1
- liboptparse segfaults with duplicate subcommand option table entries
- docker-run-systest.sh does not work anymore
- nodes are drained when a user aborts a run request with prolog running HOT 11
- flux-shell: ERROR: output: shell_output_write: Function not implemented
- when a user aborts a job early, the prolog script may get SIGTERM
- make all jobs "waitable" HOT 1
- tracking issue: standby/preemptible jobs HOT 1
- idea: use host constraint for queues instead of properties
- TOSS 4 non-TCE openmpi: Failed to open drm root directory /sys/class/drm.: No such file or directory HOT 3
- job shell blocks at exit in degraded job HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flux-core.