Giter VIP home page Giter VIP logo

kernel-tools's Introduction

Kernel Config

Build Status

https://community.cloud.online.net/t/official-linux-kernel-new-modules-optimizations-hacks/226

The kernel is built with the official mainline kernel, here are the .config files used.

The list of Scaleway's bootscripts is available at http://devhub.scaleway.com/#/bootscripts.

Modifications

We added kernel module to simulate some virtualization features:

  • serial console activation
  • remote soft reset trigger

How to build a custom kernel module

You'll need the usual toolchain for kernel compilation, which on Ubuntu is generally fulfilled by apt-get install build-essential libssl-dev. Then, the following script can be run.

# Determine versions
arch="$(uname -m)"
release="$(uname -r)"
upstream="${release%%-*}"
local="${release#*-}"

# Get kernel sources
mkdir -p /usr/src
wget -O "/usr/src/linux-${upstream}.tar.xz" "https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${upstream}.tar.xz"
tar xf "/usr/src/linux-${upstream}.tar.xz" -C /usr/src/
ln -fns "/usr/src/linux-${upstream}" /usr/src/linux
ln -fns "/usr/src/linux-${upstream}" "/lib/modules/${release}/build"

# Prepare kernel
zcat /proc/config.gz > /usr/src/linux/.config
printf 'CONFIG_LOCALVERSION="%s"\nCONFIG_CROSS_COMPILE=""\n' "${local:+-$local}" >> /usr/src/linux/.config
wget -O /usr/src/linux/Module.symvers "http://mirror.scaleway.com/kernel/${arch}/${release}/Module.symvers"
apt-get install -y libssl-dev # adapt to your package manager
make -C /usr/src/linux prepare modules_prepare

Then you can make your module as usual by configuring KDIR=/lib/modules/$(uname -r)/build/

Alternatively, you can now build a DKMS-based kernel module, for instance: apt-get install zfsutils-linux

Kernels

Name Maintainer Sources Target Links
3.2.34 Marvell Closed C1 n/a
3.18.20 Linux community Open C1 Sources
3.19.8 Linux community Open C1 Sources
4.1.6 Linux community Open C1 Sources
4.2 Linux community Open C1 Sources

Build a custom kernel using Docker

Run a make menuconfig for 3-18-std/.config

make menuconfig KERNEL=3.18-std

Build a kernel with 3.17-std/.config file

make build KERNEL=3.17-std

Advanced options

Create a new 3.10-new/.config file from scratch for kernel 3.10

make create defconfig KERNEL=3.10-new

Make oldconfig a 3.18-std kernel

make oldconfig KERNEL=3.18-std

Run a shell in the container for easy debugging and run custom commands

make shell KERNEL=3.17-std

Test a kernel with QEMU

You should use a config file made for versatile.

Build a 3.18 kernel for versatile:

make build KERNEL=3.18-defconfig_versatile

Run the kernel in qemu

make qemu KERNEL=3.18-defconfig_versatile

How to upgrade a kernel

An example with 4.0.8-docker

You should move the directory

git mv 4.0.8-docker 4.0.9-docker

Run a make oldconfig with the newest version

make oldconfig KERNEL=4.0.9-docker

Build a custom kernel from scratch (without Docker)

Prerequisites:

  • An arm(hf) compiler:
    • a cross-compiler on non-armhf host, ie gcc-arm-linux-gnueabihf
    • a standard compiler from an armhf host (you can build a kernel from your C1)
  • Theses packages: git, wget, make

Steps:

  • Configure environment

export VERSION=3.17 export ARCH=arm export ARTIFACTS=artifacts


- Download archive via web
  ```bash
wget https://kernel.org/pub/linux/kernel/v3.x/linux-$VERSION.tar.xz && tar xf linux-$VERSION.tar.xz

or via git

git clone -b v$VERSION --single-branch git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux-$VERSION
  • Generate a base .config file by building it

make ARCH=arm mvebu_v7_defconfig

  or by fetching our one

wget -O .config https://raw.githubusercontent.com/scaleway/kernel-tools/master/$VERSION/.config


- Tune the .config
  ```bash
make ARCH=arm menuconfig
# ... configure using console interface
  • Building kernel and modules

make -j $(echo nproc ' * 2' | bc) uImage modules LOADADDR=0x8000


- Export the artifacts (kernel, header, modules) to `$ARTIFACTS` directory
  ```bash
mkdir -p $ARTIFACTS
cp arch/arm/boot/uImage $ARTIFACTS/
cp System.map $ARTIFACTS/
cp .config $ARTIFACTS/
make headers_install INSTALL_HDR_PATH=$ARTIFACTS/ > /dev/null
find $ARTIFACTS/include -name ".install" -or -name "..install.cmd" -delete
make modules_install INSTALL_MOD_PATH=$ARTIFACTS/ > /dev/null
rm -rf $ARTIFACTS/modules && \
   mv $ARTIFACTS/lib/modules $ARTIFACTS && \
   rmdir $ARTIFACTS/lib && \
   rm $ARTIFACTS/modules/*/source $ARTIFACTS/modules/*/build

Minimal configuration for C1 servers

- Networking support
  - Networking options
    - 802.1Q/802.1ad VLAN Support -> **YES**
    - Packet socket -> **YES**
    - Unix domain sockets -> **YES**
- Device Drivers
  - Network device support
    - PHY Device support and infrastructure
      - Driver for MDIO Bus/PHY emulation with fixed speed/link PHYs -> **YES**
  - Block devices
    - Network block device support -> **YES**
- Kernel hacking
  - Kernel low-level debugging functions -> **YES**
  - Early prink -> **YES**
- File systems
  - The Extended 4 (ext4) filesystem -> **YES**

How to bump multiple kernels at once

# It's an example to bump the kernels from 4.5.7 to 4.6.4
git mv x86_64/4.{5.7,6.4}-std
git mv x86_64/4.{5.7,6.4}-docker
git mv x86_64/4.{5.7,6.4}-apparmor
git mv x86_64/4.{5.7,6.4}-coreos
git mv x86_64/4.{5.7,6.4}-fedora
git mv x86_64/4.{5.7,6.4}-rancher
git mv armv7l/4.{5.7,6.4}-std
git mv armv7l/4.{5.7,6.4}-apparmor
git mv armv7l/4.{5.7,6.4}-debug
git mv armv7l/4.{5.7,6.4}-docker
git mv armv7l/4.{5.7,6.4}-fedora
for kernel in `find x86_64 -name "4.6.4-*"`; do make oldconfig KERNEL=$kernel; done
for kernel in `find armv7l -name "4.6.4-*"`; do make oldconfig KERNEL=$kernel; done

How to bump a kernel

git mv x86_64/4.{5.7,6.4}-std
make oldconfig KERNEL=x86_64/4.6.4-std

Licensing

© 2014-2015 Scaleway - MIT License.

kernel-tools's People

Contributors

aimxhaisse avatar bchatelard avatar furusiyya avatar jannic avatar jmthackett avatar joelisaa avatar kwizart avatar moul avatar newtoncorp avatar quentinperez avatar rushmorem avatar superseed avatar tbillon avatar zx2c4 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kernel-tools's Issues

Missing CONFIG_XFRM_USER option

IPSec with OpenSwan requires use of xfrm_user, which is controlled by the CONFIG_XFRM_USER kernel option. Enabling this will allow the use of IPSec VPNs on online.net

Replace `make uinstall`

make uinstall seems to not be reliable across the 3.10 and 3.18, we should remove it and manually copy the uImage, config, vmlinuz

Make a 3.14 std kernel

goals :

  • to have dtb for 3.14
  • for benchmarking (cc @aimxhaisse)
  • for opensuse testing (cc @xeoncore)

todo:

  • get a 3.14 kernel-builder
  • wait for docker trusted build to finish
  • create a .config file
  • build
  • toggle-back travis checking for this kernel

Measure the stability of kernels without hardware IO/Coherency

Stable versions 3.17.8 and 3.18.5 have been released with a fix that disables the Marvell's hardware IO/Coherency feature

It seems that random problems appeared when bumping to this kernel versions

We will probably skip stable versions until the a patch re-enable it


commit 1f20756ce695ee56c2899e95757497d9c1cc8bbb
Author: Thomas Petazzoni <[email protected]>
Date:   Fri Jan 16 17:11:27 2015 +0100

    ARM: mvebu: completely disable hardware I/O coherency

    commit 8f1e8ee28660018a935c7576b9af8ffe1feab54c upstream.

    The current hardware I/O coherency is known to cause problems with DMA
    coherent buffers, as it still requires explicit I/O synchronization
    barriers, which is not compatible with the semantics expected by the
    Linux DMA coherent buffers API.

    So, in order to have enough time to validate a new solution based on
    automatic I/O synchronization barriers, this commit disables hardware
    I/O coherency entirely. Future patches will re-enable it.

    Signed-off-by: Thomas Petazzoni <[email protected]>
    Signed-off-by: Andrew Lunn <[email protected]>
    Signed-off-by: Greg Kroah-Hartman <[email protected]>

cc @tpetazzoni

4.0 crashes

I tried to reboot into 4.0. It started some selinux related relabeling stuff, but then it crashed: http://pastebin.com/KwBhsnAk . I couldn't get the whole thing, a longer scroll-back in the web console would be appreciated.

[POSTPONED] Put `ccache` on a tmpfs

To create a persistent tmpfs, we can try to :

  • mount a tmpfs
  • rsync the persistent directory to the tmpfs
  • docker run with the tmpfs as a volume
  • rsync back the tmpfs to the persistent directory

Fixup-adjtimex-freq-validation-on-32bit-systems.patch

There is an issue with current 32bit kernel (either armhfp or x86) with ntp:

févr. 22 12:51:40 localhost chronyd[1244]: Fatal error : adjtimex failed for set_frequency, freq_ppm=2.6953e+04 required_freq=4.7441e+01 required_tick=9730

This is currently pending patch upstream (seems to be fixed by 4.0-rc1)
http://pkgs.fedoraproject.org/cgit/kernel.git/commit/?h=f20&id=a80685f4b861c22000c656d87314f41cc27a66c4

This patch is currently applied on 3.18-7 kernels that are the ones used on fedora 20/21

[UNFIXED] Support SYSTEM_TRUSTED_KEYRING

root@90aa0a74e9fb:/usr/src/linux-3.18# ccache make uImage
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: 'include/generated/mach-types.h' is up to date.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
kernel/Makefile:133: *** No X.509 certificates found ***
  CHK     kernel/config_data.h
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/zImage is ready
  Image arch/arm/boot/uImage is ready

ifeq ($(CONFIG_SYSTEM_TRUSTED_KEYRING),y)
X509_CERTIFICATES-y := $(wildcard *.x509) $(wildcard $(srctree)/*.x509)
X509_CERTIFICATES-$(CONFIG_MODULE_SIG) += $(objtree)/signing_key.x509
X509_CERTIFICATES-raw := $(sort $(foreach CERT,$(X509_CERTIFICATES-y), \
                                $(or $(realpath $(CERT)),$(CERT))))
X509_CERTIFICATES := $(subst $(realpath $(objtree))/,,$(X509_CERTIFICATES-raw))

ifeq ($(X509_CERTIFICATES),)
$(warning *** No X.509 certificates found ***)
endif

ifneq ($(wildcard $(obj)/.x509.list),)
ifneq ($(shell cat $(obj)/.x509.list),$(X509_CERTIFICATES))
$(info X.509 certificate list changed)
$(shell rm $(obj)/.x509.list)
endif
endif

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.