microshift-backend-e2e

A wrapper on top of MicroShift upstream e2e tests to be run on hosts with a running MicroShift cluster.

Overview

The container is based on deliverest, which handles the remote execution.

Usage

PWD should not be a major directory like ${HOME} or similar, to avoid the following error:

Error: SELinux relabeling of /home/cloud-user is not allowed

So create a custom subdir somewhere, then run everything from it. Furthermore, PWD should contain id_rsa (and eventually also pull-secret; right now this is assumed to be on the target host already, just like a custom bundle).

Windows amd64

For debug purposes (e.g. when run outside of the sanitized environment ensured by the pipeline), use --pull=always policy for podman run. Otherwise you might end up running an old local image instead of pulling a fresh one.

TARGET_FOLDER=ms-backend-e2e
USER=crcqe
HOST=windows-crcqe.tpb.lab.eng.brq.redhat.com
BUNDLE_PATH="C:/Users/crcqe/Downloads/crc_microshift_hyperv_4.13.9_amd64.crcbundle"

podman run --pull=always --network=host --rm -it --name microshift-backend-e2e \
    -e TARGET_HOST=${HOST} \
    -e TARGET_HOST_USERNAME=${USER} \
    -e TARGET_HOST_KEY_PATH=/data/id_rsa \
    -e TARGET_FOLDER=${TARGET_FOLDER} \
    -e TARGET_RESULTS=junit/junit*.xml \
    -e OUTPUT_FOLDER=/data \
    -v $PWD:/data:z \
    -v $PWD/pullsecret:/opt/ms-backend-e2e/pullsecret:z \
    quay.io/rhqp/microshift-backend-e2e:v4.13.9-windows-amd64 \
        ms-backend-e2e/run.ps1 \
            -targetFolder ${TARGET_FOLDER} \
            -junitResultsPath ${TARGET_FOLDER}/junit \
            -pullSecretFile ${TARGET_FOLDER}/pullsecret \
            -bundlePath ${BUNDLE_PATH}

darwin amd64

TARGET_FOLDER=ms-backend-e2e
USER=crcqe
HOST=macmini-crcqe-2.tpb.lab.eng.brq.redhat.com
BUNDLE_PATH="/Users/${USER}/Downloads/crc_microshift_vfkit_4.13.9_amd64.crcbundle"

podman run --pull=always --network=host --rm -it --name microshift-backend-e2e-darwin \
    -e TARGET_HOST=${HOST} \
    -e TARGET_HOST_USERNAME=${USER} \
    -e TARGET_HOST_KEY_PATH=/data/id_rsa \
    -e TARGET_FOLDER=${TARGET_FOLDER} \
    -e TARGET_RESULTS=junit/junit*.xml \
    -e OUTPUT_FOLDER=/data \
    -v $PWD:/data:z \
    -v $PWD/pullsecret:/opt/ms-backend-e2e/pullsecret:z \
    quay.io/rhqp/microshift-backend-e2e:v4.13.9-darwin-amd64 \
        ms-backend-e2e/run.sh \
            -t ${TARGET_FOLDER} \
            -p ${TARGET_FOLDER}/pullsecret \
            -r ${TARGET_FOLDER}/junit \
            -b ${BUNDLE_PATH}

linux amd64

TARGET_FOLDER=ms-backend-e2e
USER=cloud-user
HOST=rhel-crcqe.tpb.lab.eng.brq.redhat.com
BUNDLE_PATH="/home/${USER}/Downloads/crc_microshift_libvirt_4.13.9_amd64.crcbundle"

podman run --pull=always --network=host --rm -it --name microshift-backend-e2e \
    -e TARGET_HOST=${HOST} \
    -e TARGET_HOST_USERNAME=${USER} \
    -e TARGET_HOST_KEY_PATH=/data/id_rsa \
    -e TARGET_FOLDER=${TARGET_FOLDER} \
    -e TARGET_RESULTS=junit/junit*.xml \
    -e OUTPUT_FOLDER=/data \
    -v $PWD:/data:z \
    -v $PWD/pullsecret:/opt/ms-backend-e2e/pullsecret:z \
    quay.io/rhqp/microshift-backend-e2e:v4.13.9-linux-amd64 \
        ms-backend-e2e/run.sh \
            -t ${TARGET_FOLDER} \
            -p ${TARGET_FOLDER}/pullsecret \
            -r ${TARGET_FOLDER}/junit \
            -b ${BUNDLE_PATH}

Updating OpenShift version

Create a new branch, e.g. v4.13.9 and add all changes needed alongside this version change.
Merge the branch into main.
Create a tag, e.g. v4.13.9 and push it to main with git push origin v4.13.9.
GH-Actions will trigger a build based on the presence of the new tag and push the new image to quay.io.

Cannot find binaries even if on the PATH

[cloud-user@rhel-crcqe ms-backend-e2e]$ ls -la ms-backend-e2e 
-rwxr-xr-x. 1 cloud-user cloud-user 280798248 Apr 27 14:11 ms-backend-e2e
[cloud-user@rhel-crcqe ms-backend-e2e]$ ./ms-backend-e2e 
-bash: ./ms-backend-e2e: No such file or directory

Issue with using alpine base image and glibc vs musl: https://stackoverflow.com/questions/66963068/docker-alpine-executable-binary-not-found-even-if-in-path.

The way out of it is one of 2 ways:

~~Maybe use apk add libc6-compat and hope it helps to run alpine containers on rhel as well.~~
Use a different base image in the containerfile:

FROM registry.access.redhat.com/ubi8/go-toolset:1.18 as builder

USER root
ARG OPENSHIFT_VERSION
ARG OS 
ARG ARCH

ENV GOOS=${OS} \
    GOARCH=${ARCH} \
    UPSTREAM=https://github.com/openshift/origin.git \
    BRANCH=release-${OPENSHIFT_VERSION}\
    EPEL=https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

RUN rpm -ivh ${EPEL} \ 
    && dnf --enablerepo=epel install -y git gcc gcc-c++ kernel-headers curl \
    && git clone --depth 1 --branch ${BRANCH} ${UPSTREAM} \
    && cd origin \
    && go build -o ./build/ms-backend-e2e -mod=vendor -trimpath github.com/openshift/origin/cmd/openshift-tests

RUN if [[ "${OS}" == "windows" ]]; then curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/windows/${ARCH}/kubectl.exe"; else \
    curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/${OS}/${ARCH}/kubectl"; fi 

FROM quay.io/rhqp/deliverest:v0.0.1

LABEL org.opencontainers.image.authors="Adrian Riobo<[email protected]>"

ENV ASSETS_FOLDER /opt/ms-backend-e2e

COPY --from=builder /opt/app-root/src/origin/build/ms-backend-e2e /opt/app-root/src/kubectl* ${ASSETS_FOLDER}/
ARG OS 
COPY /lib/${OS}/* /lib/common/* ${ASSETS_FOLDER}/
RUN chmod +x ${ASSETS_FOLDER}/run.* \
    && chmod +x ${ASSETS_FOLDER}/kubectl*
COPY /hooks /

jsliacan / microshift-backend-e2e Goto Github PK