theopenlab / openlab Goto Github PK

We have a number of academic supporting organizations who offer scriptable methods to spin up bare metal nodes which OpenStack or other tools/applications can be deployed/tested on (we can only use them for a specific amount of time)

ChameleonCloud
Emulab
OneLab
CloudLab
GENI

In Kubernetes/OpenStack integration scenario, we should keep pipeline name, job name and trigger comment consistence, that build consist user experience for developer from k8s community, and easy to understand how to trigger the specific jobs when the job failed.

• update all of pipeline name, job name and trigger comment for k8s jobs
• run all of jobs in separate pipeline when PR commit and update, and report to github independently
• branch-v1.10.0 should be just branch-v1.10
• update https://github.com/theopenlab/cicd-howto#kubernetescloud-provider-openstack

• openlab-jobs

This is similar to #12

dims/openstack-cloud-controller-manager#34

Right now we set the version of ruby to install via vars in the .zuul.yaml file ( ex: https://github.com/flystack/misty/blob/master/.zuul.yaml#L14 ) and we need to instead do this via inherited job process ( ex: https://github.com/openstack-infra/openstack-zuul-jobs/blob/c389cfd3ccafe3dafb09063fd137d55c564e41d6/zuul.d/jobs.yaml#L249-299 )

We should refactor our code so that one or many versions of Ruby can be installed without needing to do this in .zuul.yaml per repo

http://bosh.io

BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. See docs.

Add systemd management scripts to utilize the systemctl utility for starting, stopping, restarting services.

TestReadConfig test case will modify the openstack credential envs, which causes the subsequent 3 test cases TestNewOpenStack/TestLoadBalancer/TestVolumes skipped at openstack credential verification [1]

[1] https://github.com/dims/openstack-cloud-controller-manager/blob/master/pkg/cloudprovider/providers/openstack/openstack_test.go#L90

https://docs.google.com/spreadsheets/d/1cdzFeV5I4Wk9FK57yqQmp5JJdGfKzEOdB3Vtt9vnVJM/edit#gid=0

• RuiChen: if we can get a public cloud account of VEXXHOST, it's easy to integrate with OpenLab nodepool.

• Melvin: provided VEXXHOST credentials to the team.

https://developer.github.com/v3/#pagination

• https://github.com/theopenlab/pkgcloud
• https://github.com/theopenlab/pkgcloud-integration-tests

• remove openstack-cloud-controller-manager-acceptance-test-* jobs
• rename openstack-cloud-controller-manager-unittest to cloud-provider-openstack-unittest-nested, mark it Deprecated

Another works have been finished when new cloud-provider-openstack-* jobs are commited.

dims/openstack-cloud-controller-manager#56

ref: kubernetes/cloud-provider-openstack/issues/122

1. Q pipeline and job define
2. gophercloud .zuul.yaml update
3. terraform .zuul.yaml update

One SDK per language focus:

Go - gophercloud
Python - python-openstacksdk
Ruby - fog-openstack
Java - jclouds
JavaScript - pkgcloud

Load balancer stuck in PENDING_DELETE status when we delete load balancer instance in vexxhost, no pool and listener is associsated with these load balancers, but we can not delete them.
OpenLab cloud-provider-openstack-acceptance-test-lb-octavia job will call vexxhost API to create load balancer when PR trigger and periodic task execute, and clean up pool, listener and load balancer instance when job finished automatically.
Quota limit of load balancer instance is 10 in current tenant, so if we can not clean up these resources, cloud-provider-openstack-acceptance-test-lb-octavia job will be failing when usage reaches the quota limit.

(neutron) lbaas-loadbalancer-list
+--------------------------------------+----------------------------------+--------------+---------------------+----------+
| id                                   | name                             | vip_address  | provisioning_status | provider |
+--------------------------------------+----------------------------------+--------------+---------------------+----------+
| 1b3eebea-feb3-46b1-85a2-70216618e9b7 | ab9c7e06c446611e8ae30fa163eb587d | 192.168.0.18 | PENDING_DELETE      | octavia  |
| b14b32d2-11bc-49b1-a6ff-5a4c2cba3a6a | a2c2d09c7432a11e88859fa163e1960d | 192.168.0.31 | PENDING_DELETE      | octavia  |
| b9684530-26e9-480a-9ea0-ecc0dd8eea40 | aabe446b643e011e89cacfa163e8f033 | 192.168.0.13 | PENDING_DELETE      | octavia  |
| c14b4043-afae-486e-a45a-f545b572dc84 | a1a13cb85375611e88910fa163ea4997 | 192.168.0.28 | PENDING_DELETE      | octavia  |
| ed255e90-9b42-46c8-8224-e85331c4fe56 | a858f0cb0435f11e88bd5fa163e99092 | 192.168.0.19 | PENDING_DELETE      | octavia  |
+--------------------------------------+----------------------------------+--------------+---------------------+----------+

• RuiChen: How we can make rally to work together with SDK, like gophercloud or terraform?

• Add separated pipeline
• Use export-vexxhost-openrc role to get vexxhost credential
• Remove post.yaml
• Update .zuul.yaml in kubernetes/cloud-provider-openstack
• Enable k8s cluster DNS
• Update table https://github.com/theopenlab/cicd-howto#kubernetescloud-provider-openstack

We need to move these to terraform-providers/terraform-provider-openstack

terraform-provider-openstack-acceptance-test
terraform-provider-openstack-acceptance-test-designate
terraform-provider-openstack-acceptance-test-fwaas
terraform-provider-openstack-acceptance-test-lbaas
terraform-provider-openstack-acceptance-test-trove
terraform-provider-openstack-unittest

• Liusheng
• Zhuli
• RuiChen
• Dolpher

https://github.com/kubernetes/cloud-provider-openstack/

General issues/concerns/changes that need to happen related to OpenLab Infrastructure.

• get OpenLab github app attached to ansible/ansible repo ( for shade-ansible-devel-functional-devstack )
• setup jobs that can run shade, openstacksdk, and python-openstackclient functional tests against older clouds and report back to openstack gerrit as a 3rd party CI
• setup jobs that run shade and openstacksdk functional tests against live public cloud endpoints

devstack-gate uses neutron-legacy by default

• At the last Friday of each month

tebeka/go2xunit#29
tebeka/go2xunit#50

Cu

Initial conversation started:

fog/fog-openstack#385

• move ci tools out of OTC and into VEXXHOST
• continue to use as nodepool

We have two additional clouds we need to add to nodepool:

SWITCH
CityNetwork

Melvin: Can we get a bit more detail here?
RuiChen: Now we use split pipeline to support "recheck stable/pike" and "recheck designate", that cause so many pipeline should be defined for testing on different versions and different services (M * N pipelines), we hope zuul can support "recheck {version} {service}" in comments, and zuul can trigger different jobs in one pipeline define.