The goresctrl library provides Go interface to manage following resources.
intel / goresctrl Goto Github PK
View Code? Open in Web Editor NEWGolang library for managing resctrl filesystem
License: Apache License 2.0
Golang library for managing resctrl filesystem
License: Apache License 2.0
The goresctrl library provides Go interface to manage following resources.
Currently we only read out a small part of the information available. We should retrieve it all (corresponding what
intel-speed-select perf-profile info
does)
My goal is to let the hotel group use 0xff0 of l3 cache on socket0, the stress group use 0x00f of l3 cache on socket0, but neither group use l3 cache on socket1. I use customed yml file to adjust my resctrl file system, but get unexptected result.
My config.yml:
options:
l3:
optional: true
partitions:
default:
l3Allocation:
0: "0xfff"
1: "0xfff"
classes:
system/default:
l3Allocation:
0: "0xfff"
1: "0xfff"
hotel:
l3Allocation:
0: "0xff0"
1: "0x0"
stress:
l3Allocation:
0: "0x00f"
1: "0x0"
log in debug:
[ rdt ] DEBUG: resolving configuration:
[ rdt ] DEBUG: options:
[ rdt ] DEBUG: l2:
[ rdt ] DEBUG: Optional: false
[ rdt ] DEBUG: l3:
[ rdt ] DEBUG: Optional: true
[ rdt ] DEBUG: mb:
[ rdt ] DEBUG: Optional: false
[ rdt ] DEBUG: partitions:
[ rdt ] DEBUG: default:
[ rdt ] DEBUG: classes:
[ rdt ] DEBUG: hotel:
[ rdt ] DEBUG: kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: l2Allocation: null
[ rdt ] DEBUG: l3Allocation:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0xff0"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0"
[ rdt ] DEBUG: mbAllocation: null
[ rdt ] DEBUG: stress:
[ rdt ] DEBUG: kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: l2Allocation: null
[ rdt ] DEBUG: l3Allocation:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0x00f"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0"
[ rdt ] DEBUG: mbAllocation: null
[ rdt ] DEBUG: system/default:
[ rdt ] DEBUG: kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: l2Allocation: null
[ rdt ] DEBUG: l3Allocation:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: mbAllocation: null
[ rdt ] DEBUG: l2Allocation: null
[ rdt ] DEBUG: l3Allocation:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Code: ""
[ rdt ] DEBUG: Data: ""
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: mbAllocation: null
But when apply this configuration to my resctrl file system, it shows that:
[ rdt ] DEBUG: applying resolved config:
[ rdt ] DEBUG: Classes:
[ rdt ] DEBUG: hotel:
[ rdt ] DEBUG: CATSchema:
[ rdt ] DEBUG: L2:
[ rdt ] DEBUG: Alloc: null
[ rdt ] DEBUG: Lvl: L2
[ rdt ] DEBUG: L3:
[ rdt ] DEBUG: Alloc:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Unified: "0xff0"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Unified: "0x1"
[ rdt ] DEBUG: Lvl: L3
[ rdt ] DEBUG: Kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: MBSchema: null
[ rdt ] DEBUG: Partition: default
[ rdt ] DEBUG: stress:
[ rdt ] DEBUG: CATSchema:
[ rdt ] DEBUG: L2:
[ rdt ] DEBUG: Alloc: null
[ rdt ] DEBUG: Lvl: L2
[ rdt ] DEBUG: L3:
[ rdt ] DEBUG: Alloc:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Unified: "0xf"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Unified: "0x1"
[ rdt ] DEBUG: Lvl: L3
[ rdt ] DEBUG: Kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: MBSchema: null
[ rdt ] DEBUG: Partition: default
[ rdt ] DEBUG: system/default:
[ rdt ] DEBUG: CATSchema:
[ rdt ] DEBUG: L2:
[ rdt ] DEBUG: Alloc: null
[ rdt ] DEBUG: Lvl: L2
[ rdt ] DEBUG: L3:
[ rdt ] DEBUG: Alloc:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: Lvl: L3
[ rdt ] DEBUG: Kubernetes:
[ rdt ] DEBUG: denyContainerAnnotation: false
[ rdt ] DEBUG: denyPodAnnotation: false
[ rdt ] DEBUG: MBSchema: null
[ rdt ] DEBUG: Partition: default
[ rdt ] DEBUG: Options:
[ rdt ] DEBUG: l2:
[ rdt ] DEBUG: Optional: false
[ rdt ] DEBUG: l3:
[ rdt ] DEBUG: Optional: true
[ rdt ] DEBUG: mb:
[ rdt ] DEBUG: Optional: false
[ rdt ] DEBUG: Partitions:
[ rdt ] DEBUG: default:
[ rdt ] DEBUG: CAT:
[ rdt ] DEBUG: L2:
[ rdt ] DEBUG: Alloc: {}
[ rdt ] DEBUG: Lvl: L2
[ rdt ] DEBUG: L3:
[ rdt ] DEBUG: Alloc:
[ rdt ] DEBUG: "0":
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: "1":
[ rdt ] DEBUG: Unified: "0xfff"
[ rdt ] DEBUG: Lvl: L3
[ rdt ] DEBUG: MB: {}
I read the code, and found that it will change 0 to 0x1
func listStrToBitmask(str string) (bitmask, error) {
b := bitmask(0)
// Empty bitmask
if len(str) == 0 {
return b, nil
}
ranges := strings.Split(str, ",")
for _, ran := range ranges {
split := strings.SplitN(ran, "-", 2)
bitNum, err := strconv.ParseUint(split[0], 10, 6)
if err != nil {
return b, fmt.Errorf("invalid bitmask %q: %v", str, err)
}
if len(split) == 1 {
b |= 1 << bitNum // it will make 0 to 0x1
} else {
endNum, err := strconv.ParseUint(split[1], 10, 6)
if err != nil {
return b, fmt.Errorf("invalid bitmask %q: %v", str, err)
}
if endNum <= bitNum {
return b, fmt.Errorf("invalid range %q in bitmask %q", ran, str)
}
b |= (1<<(endNum-bitNum+1) - 1) << bitNum
}
}
return b, nil
}
Is this a bug or a feature, and if it is a bug, can I try to fix it?
Test the parts where it makes sense โ we're quite heavily dependent on the HW
In #59 we merged a simple (almost a sketch) command line utility for SST.
A similar too would be useful for RDT as well. It would serve as a reference implementation for using the API and would also be useful for debugging and testing.
It'd be nice to have some documentation about the blockio API, e.g. in the readme.
Decent "docstring" documentation of the API, with a package-level description, too
Support enabling/disabling SST-CP, configuring the CLOSes and assigning CPUs to CLOSes.
in pkg/rdt/rdt.go
:
RootClassName = "system/default"
whereas in doc/rdt.md
it's DEFAULT
.
It might be useful for the user to be able to configure the amount of cache allocation in kiB/MiB in addition to percentages or absolute bitmasks.
Possible future work. Support similar higher level management through config file, similar to what rdt and blockio has.
The root (or default) resctrl group can be configured by specifying class with the name system/default
in the RDT config. Unfortunately, this is not currently documented anywhere (except for the source code ๐)
In #59 we implemented a simple (almost a sketch) command line utility for SST.
A similar too would be useful for blockio as well. It would serve as a reference implementation for using the API and would also be useful for debugging and testing.
Nice to have would be a flag in the config to request/guarantee that a class has exclusive cache allocation.
The behavior would just be a bit blunt (I think): configuration will fail if the class cannot be made exclusive. Reasons might be RDT misconfiguration (in goresctrl rdt config) or that there are other existing classes overlapping (that are not managed by goresctrl)
Make it possible to enable/disable SST-TF and configure the set of high-priority CPUs.
Currently, we require that allocation is explicitly specified on the partition level. A more user friendly way would probably be to assume 100% if nothing is specified, especially because many usage scenarios seem to only involve one partition, in which case it seems dump to require to explicitly specify 100% allocations for all resources.
This might be lead to surprises/slightly misleading error messages when multiple partitions are specified: Cache allocations on partition level are exclusive and cannot overlap. Basically with multiple partitions an error occurs if any partition requests 100% of cache lines. But probably that can be mitigated by improving error messages.
An example:
partitions:
default:
classes:
...
would imply
partitions:
default:
l2Allocation: "100%"
l3Allocation: "100%"
mbAllocation: ["100%", 4294967295]
classes:
...
Add a README describing the technology and usage of the package. Something similar we have for RDT, for example.
Use case: as a k8s cluster admin I want to define the default I/O scheduler priority and throttling for all pods that are not annotated.
Need for the feature raised by @pacoxu in Kubernetes issue 92287.
A possible solution on both RDT and blockio class configuration file:
DefaultClass: UnannotatedContainers
Classes:
UnannotatedContainers:
...
ClassA:
...
...
pkg/rdt/README.md
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.