intel / goresctrl Goto Github PK

View Code? Open in Web Editor NEW

43.0 5.0 16.0 458 KB

Golang library for managing resctrl filesystem

License: Apache License 2.0

Go 99.56% Makefile 0.14% Shell 0.30%

goresctrl's Introduction

Go Resource Control

The goresctrl library provides Go interface to manage following resources.

CPU cache allocation and memory bandwidth, see the rdt (Intel Resource Director Technology) package.
CPU frequency in core granularity, see the sst (Intel Speed Select Technology) package.
Storage I/O scheduler priority and bandwidth, see the blockio package.

goresctrl's People

Contributors

Stargazers

Watchers

Forkers

marquiz klihub isabella232 askervin jukkar okartau kitianfresh pauldintel mythi sergeimonakhov yuanqijing rouzip yanjing1104 wpross lmoyalrh advanced-memory-analytics

goresctrl's Issues

sst: support configuring SST-PP

sst: full read-only support

Currently we only read out a small part of the information available. We should retrieve it all (corresponding what
intel-speed-select perf-profile info
does)

question about min llc allocation

My goal is to let the hotel group use 0xff0 of l3 cache on socket0, the stress group use 0x00f of l3 cache on socket0, but neither group use l3 cache on socket1. I use customed yml file to adjust my resctrl file system, but get unexptected result.

My config.yml:

options:
  l3:
    optional: true
partitions:
  default:
    l3Allocation:
      0: "0xfff"
      1: "0xfff"
    classes:
      system/default:
        l3Allocation:
          0: "0xfff"
          1: "0xfff"
      hotel:
        l3Allocation:
          0: "0xff0"
          1: "0x0"
      stress:
        l3Allocation:
          0: "0x00f"
          1: "0x0"

[ rdt ] DEBUG: resolving configuration:
[ rdt ] DEBUG:   options:
[ rdt ] DEBUG:     l2:
[ rdt ] DEBUG:       Optional: false
[ rdt ] DEBUG:     l3:
[ rdt ] DEBUG:       Optional: true
[ rdt ] DEBUG:     mb:
[ rdt ] DEBUG:       Optional: false
[ rdt ] DEBUG:   partitions:
[ rdt ] DEBUG:     default:
[ rdt ] DEBUG:       classes:
[ rdt ] DEBUG:         hotel:
[ rdt ] DEBUG:           kubernetes:
[ rdt ] DEBUG:             denyContainerAnnotation: false
[ rdt ] DEBUG:             denyPodAnnotation: false
[ rdt ] DEBUG:           l2Allocation: null
[ rdt ] DEBUG:           l3Allocation:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0xff0"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0"
[ rdt ] DEBUG:           mbAllocation: null
[ rdt ] DEBUG:         stress:
[ rdt ] DEBUG:           kubernetes:
[ rdt ] DEBUG:             denyContainerAnnotation: false
[ rdt ] DEBUG:             denyPodAnnotation: false
[ rdt ] DEBUG:           l2Allocation: null
[ rdt ] DEBUG:           l3Allocation:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0x00f"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0"
[ rdt ] DEBUG:           mbAllocation: null
[ rdt ] DEBUG:         system/default:
[ rdt ] DEBUG:           kubernetes:
[ rdt ] DEBUG:             denyContainerAnnotation: false
[ rdt ] DEBUG:             denyPodAnnotation: false
[ rdt ] DEBUG:           l2Allocation: null
[ rdt ] DEBUG:           l3Allocation:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Code: ""
[ rdt ] DEBUG:               Data: ""
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:           mbAllocation: null
[ rdt ] DEBUG:       l2Allocation: null
[ rdt ] DEBUG:       l3Allocation:
[ rdt ] DEBUG:         "0":
[ rdt ] DEBUG:           Code: ""
[ rdt ] DEBUG:           Data: ""
[ rdt ] DEBUG:           Unified: "0xfff"
[ rdt ] DEBUG:         "1":
[ rdt ] DEBUG:           Code: ""
[ rdt ] DEBUG:           Data: ""
[ rdt ] DEBUG:           Unified: "0xfff"
[ rdt ] DEBUG:       mbAllocation: null

But when apply this configuration to my resctrl file system, it shows that:

[ rdt ] DEBUG: applying resolved config:
[ rdt ] DEBUG:   Classes:
[ rdt ] DEBUG:     hotel:
[ rdt ] DEBUG:       CATSchema:
[ rdt ] DEBUG:         L2:
[ rdt ] DEBUG:           Alloc: null
[ rdt ] DEBUG:           Lvl: L2
[ rdt ] DEBUG:         L3:
[ rdt ] DEBUG:           Alloc:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Unified: "0xff0"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Unified: "0x1"
[ rdt ] DEBUG:           Lvl: L3
[ rdt ] DEBUG:       Kubernetes:
[ rdt ] DEBUG:         denyContainerAnnotation: false
[ rdt ] DEBUG:         denyPodAnnotation: false
[ rdt ] DEBUG:       MBSchema: null
[ rdt ] DEBUG:       Partition: default
[ rdt ] DEBUG:     stress:
[ rdt ] DEBUG:       CATSchema:
[ rdt ] DEBUG:         L2:
[ rdt ] DEBUG:           Alloc: null
[ rdt ] DEBUG:           Lvl: L2
[ rdt ] DEBUG:         L3:
[ rdt ] DEBUG:           Alloc:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Unified: "0xf"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Unified: "0x1"
[ rdt ] DEBUG:           Lvl: L3
[ rdt ] DEBUG:       Kubernetes:
[ rdt ] DEBUG:         denyContainerAnnotation: false
[ rdt ] DEBUG:         denyPodAnnotation: false
[ rdt ] DEBUG:       MBSchema: null
[ rdt ] DEBUG:       Partition: default
[ rdt ] DEBUG:     system/default:
[ rdt ] DEBUG:       CATSchema:
[ rdt ] DEBUG:         L2:
[ rdt ] DEBUG:           Alloc: null
[ rdt ] DEBUG:           Lvl: L2
[ rdt ] DEBUG:         L3:
[ rdt ] DEBUG:           Alloc:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:           Lvl: L3
[ rdt ] DEBUG:       Kubernetes:
[ rdt ] DEBUG:         denyContainerAnnotation: false
[ rdt ] DEBUG:         denyPodAnnotation: false
[ rdt ] DEBUG:       MBSchema: null
[ rdt ] DEBUG:       Partition: default
[ rdt ] DEBUG:   Options:
[ rdt ] DEBUG:     l2:
[ rdt ] DEBUG:       Optional: false
[ rdt ] DEBUG:     l3:
[ rdt ] DEBUG:       Optional: true
[ rdt ] DEBUG:     mb:
[ rdt ] DEBUG:       Optional: false
[ rdt ] DEBUG:   Partitions:
[ rdt ] DEBUG:     default:
[ rdt ] DEBUG:       CAT:
[ rdt ] DEBUG:         L2:
[ rdt ] DEBUG:           Alloc: {}
[ rdt ] DEBUG:           Lvl: L2
[ rdt ] DEBUG:         L3:
[ rdt ] DEBUG:           Alloc:
[ rdt ] DEBUG:             "0":
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:             "1":
[ rdt ] DEBUG:               Unified: "0xfff"
[ rdt ] DEBUG:           Lvl: L3
[ rdt ] DEBUG:       MB: {}

I read the code, and found that it will change 0 to 0x1

func listStrToBitmask(str string) (bitmask, error) {
	b := bitmask(0)

	// Empty bitmask
	if len(str) == 0 {
		return b, nil
	}

	ranges := strings.Split(str, ",")
	for _, ran := range ranges {
		split := strings.SplitN(ran, "-", 2)

		bitNum, err := strconv.ParseUint(split[0], 10, 6)
		if err != nil {
			return b, fmt.Errorf("invalid bitmask %q: %v", str, err)
		}

		if len(split) == 1 {
			b |= 1 << bitNum // it will make 0 to 0x1
		} else {
			endNum, err := strconv.ParseUint(split[1], 10, 6)
			if err != nil {
				return b, fmt.Errorf("invalid bitmask %q: %v", str, err)
			}
			if endNum <= bitNum {
				return b, fmt.Errorf("invalid range %q in bitmask %q", ran, str)
			}
			b |= (1<<(endNum-bitNum+1) - 1) << bitNum
		}
	}
	return b, nil
}

Is this a bug or a feature, and if it is a bug, can I try to fix it?

sst: unit tests

Test the parts where it makes sense – we're quite heavily dependent on the HW

rdt: command line utility

In #59 we merged a simple (almost a sketch) command line utility for SST.

A similar too would be useful for RDT as well. It would serve as a reference implementation for using the API and would also be useful for debugging and testing.

Documentation for blockio

It'd be nice to have some documentation about the blockio API, e.g. in the readme.

sst: API documentation

Decent "docstring" documentation of the API, with a package-level description, too

sst: support configuring SST-CP

Support enabling/disabling SST-CP, configuring the CLOSes and assigning CPUs to CLOSes.

RootClassName mismatch in doc vs. code

in pkg/rdt/rdt.go:
RootClassName = "system/default"

whereas in doc/rdt.md it's DEFAULT.

rdt: support cache allocation in kiB/MiB

It might be useful for the user to be able to configure the amount of cache allocation in kiB/MiB in addition to percentages or absolute bitmasks.

sst: config file support

Possible future work. Support similar higher level management through config file, similar to what rdt and blockio has.

rdt: document system/default class

The root (or default) resctrl group can be configured by specifying class with the name system/default in the RDT config. Unfortunately, this is not currently documented anywhere (except for the source code 🙄)

blockio: command line utility

In #59 we implemented a simple (almost a sketch) command line utility for SST.

A similar too would be useful for blockio as well. It would serve as a reference implementation for using the API and would also be useful for debugging and testing.

rdt: config flag for a class to reqeust exclusive cache allocation

Nice to have would be a flag in the config to request/guarantee that a class has exclusive cache allocation.

The behavior would just be a bit blunt (I think): configuration will fail if the class cannot be made exclusive. Reasons might be RDT misconfiguration (in goresctrl rdt config) or that there are other existing classes overlapping (that are not managed by goresctrl)

sst: support configuring SST-TF

Make it possible to enable/disable SST-TF and configure the set of high-priority CPUs.

sst: support enabling SST-BF

rdt: partition level defaults in config

Currently, we require that allocation is explicitly specified on the partition level. A more user friendly way would probably be to assume 100% if nothing is specified, especially because many usage scenarios seem to only involve one partition, in which case it seems dump to require to explicitly specify 100% allocations for all resources.

This might be lead to surprises/slightly misleading error messages when multiple partitions are specified: Cache allocations on partition level are exclusive and cannot overlap. Basically with multiple partitions an error occurs if any partition requests 100% of cache lines. But probably that can be mitigated by improving error messages.

An example:

partitions:
  default:
    classes:
      ...

would imply

partitions:
  default:
    l2Allocation: "100%"
    l3Allocation: "100%"
    mbAllocation: ["100%", 4294967295]
    classes:
      ...

sst: add README

Add a README describing the technology and usage of the package. Something similar we have for RDT, for example.

RDT & blockio: enable defining parameters for unannotated containers

Use case: as a k8s cluster admin I want to define the default I/O scheduler priority and throttling for all pods that are not annotated.

Need for the feature raised by @pacoxu in Kubernetes issue 92287.

A possible solution on both RDT and blockio class configuration file:

DefaultClass: UnannotatedContainers
Classes:
    UnannotatedContainers:
        ...
    ClassA:
        ...
    ...

documentation: improve README

improve the general description of the library
-split out the RDT parts into a separate pkg/rdt/README.md

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.