Comments (4)
There really are two categories of commands here. Consider the different command line parameters that apply to each scenario.
- One-time (or rare) disk configuration. Requires privileged access.
- Long-term orchestration. Should not require privileged access.
For the #2, you wouldn't expect to specify devices every time you launch the orchestrator. You would only expect to specify devices for #1, then the orchestrator would use them thereafter. If the parameters for #1 are passed during every run of the orchestrator (#2), you can run into issues. The same parameters may not have the same desired effect.
For example, there are potentially issues if you run this command at every boot: castled --devices=/dev/sdd
. After reboot, sdd
could represent a different disk. Under the covers we are using serial number for the disk so we are fine, but the user is passing a name that could change meaning.
- If the disk is unchanged, why should the parameter be passed again on subsequent launches? It is already configured.
- If the disk represents a different disk, should we start using the new disk? Should we destroy the osd on the old disk that moved? What is the user intention?
To eliminate the confusion here, it seems we need two distinct commands that don't overlap.
sudo casteld prepare --data-devices=sdb,sdc
.- Preparing devices is done explicitly then the process exits.
castled
- long-running orchestration will use the devices that have already been provisioned. No confusion with the arguments on subsequent runs and no privileges necessary for the long-running process.
This means we would need to store state on the system partition such as in /var/lib/castle
. The provisioning of disks will write the expected configuration there, and the orchestrator will read the state. Other config tied to this machine (such as a unique machine id
) can be persisted to that folder as well.
from rook.
I think you're proposing that we get rid of auto-prepare mode and force users to call castled prepare
to cleanly separate between code that requires root privs and code that does not. Also it would be nice if castled prepare performed no network operations at all.
the disk identification issue seems independent and we should probably enable letting users include and exclude devices use linux persistent disk names.
I'm unclear how new devices get added if we used the explicit prepare step.
from rook.
here's the patch to invoke libblkid from cgo:
commit b07e06e8f6063d8e8d63aacc2738c917cb42c662
Author: Bassam Tabbara <[email protected]>
Date: Fri Oct 14 13:48:09 2016 -0700
prototype lsblk using cgo
diff --git a/cmd/blkid/main.go b/cmd/blkid/main.go
new file mode 100644
index 0000000..655feaa
--- /dev/null
+++ b/cmd/blkid/main.go
@@ -0,0 +1,16 @@
+package main
+
+import (
+ "fmt"
+ "os"
+
+ "github.com/quantum/castle/pkg/system"
+)
+
+func main() {
+ value, err := system.LookupProbeValue(os.Args[1], os.Args[2])
+ if err != nil {
+ panic(err)
+ }
+ fmt.Printf(value)
+}
diff --git a/pkg/system/block_linux.go b/pkg/system/block_linux.go
new file mode 100644
index 0000000..455e662
--- /dev/null
+++ b/pkg/system/block_linux.go
@@ -0,0 +1,35 @@
+package system
+
+// #include <errno.h>
+// #include <blkid/blkid.h>
+// #cgo LDFLAGS: -lblkid -luuid
+import "C"
+
+import (
+ "fmt"
+ "unsafe"
+)
+
+func LookupProbeValue(device, label string) (string, error) {
+ var blkidProbe C.blkid_probe
+ var err C.int
+
+ blkidProbe = C.blkid_new_probe_from_filename(C.CString(device))
+ if blkidProbe == nil {
+ return "", fmt.Errorf("failed to create new blkid probe for device %s", device)
+ }
+ defer C.blkid_free_probe(blkidProbe)
+
+ err = C.blkid_do_probe(blkidProbe)
+ if err != 0 {
+ return "", fmt.Errorf("failed to probe device %s", device)
+ }
+
+ var bufPtr *C.char
+ err = C.blkid_probe_lookup_value(blkidProbe, C.CString(label), (**C.char)(unsafe.Pointer(&bufPtr)), nil)
+ if err != 0 {
+ return "", fmt.Errorf("failed to lookup probe value %s for device %s", label, device)
+ }
+
+ return C.GoString(bufPtr), nil
+}
diff --git a/pkg/system/block_linux_test.go b/pkg/system/block_linux_test.go
new file mode 100644
index 0000000..8f95c44
--- /dev/null
+++ b/pkg/system/block_linux_test.go
@@ -0,0 +1,10 @@
+package system
+
+import "testing"
+
+func TestLookupValue(t *testing.T) {
+ _, err := LookupProbeValue("/dev/sda1", "UUID")
+ if err != nil {
+ t.Fatal(err)
+ }
+}
from rook.
closing this in favor of #400.
from rook.
Related Issues (20)
- Add Support for Kafka and Elasticsearch database application on Rook clusters HOT 2
- Automatically decide type of MDS pod scaling HOT 1
- ci: usage of depricated dependencies in github actions workflows HOT 2
- osd stuck after hearbeat failure HOT 6
- ceph logs are not pruned/cleaned up for components that are no longer scheduled on a node HOT 13
- Linting jobs are failing after pylint update HOT 2
- Publishing images is failing since updating the CI to Ubuntu 22.04 HOT 2
- bucket policy is overwriten on external ceph cluster. HOT 2
- rook Failed to connect to the external object storage of the cluster HOT 12
- Fix suppressed pylint errors
- rook-ceph-tools does not have the s5cmd command HOT 7
- external ceph cluster should watch for updated configuration
- rook rgw failed to apply for pvc HOT 7
- MDS behind on trimming every 4-5 weeks causing issue for ceph filesystem HOT 2
- During the update from 1.8 to 1.9 the operator started crashing HOT 21
- Dont create the Fencing CR when CSI is disabled
- How do I verify that my rook-ceph is using bluestore HOT 6
- Failed to start clean up jobs for SubvolumeGroup and Radosnamespace resource
- SVG cleanup jobs fails due to missing env variable.
- Failure to create OSD (obtaining uniq OSD.id) blocks further provisioning and possibly does not reach desired multiple OSDs per device HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rook.