att-comdev / cicd Goto Github PK

The pegleg repo github.com/att-comdev/pegleg needs to support artifact build and deployment on merges. - The Docker image for the tool should be built and published into the standard repositories.

• Documentation should be rendered and published in readthedocs.io with other UCP documentation

There is a bug in superseed.sh script:

Problem:
apt retrieves changelogs for packages from changelogs.ubuntu.com and can find changelogs for packages that are built by Ubuntu team. apt is unable to find changelog data on website mentioned above for packages we re-build or backport for the project.

Desired behavior:
apt is able to show proper changelog for custom built packages as well as for stock packages

Steps to reproduce:

1. optional: setup Artifactory as repo
2. add ubuntu-packages repo
3. optional: update/install iproute2 ethtool i40e-dkms packages, they will be updated from Artifactory ubuntu-packages repo
4. apt changelog iproute2 ethtool i40e-dkms command will fail to retrieve changelogs

Proposed solution:

1. set up Artifactory to generate changelog repo similar to the one Ubuntu serves (at least for repositories with our custom packages)
2. set up apt to retrieve changelogs from artifactory

Setup:

export artf_url="https://artifacts-aic.atlantafoundry.com/artifactory"
export artf_custom_repo="${artf_url}/ubuntu-packages"
export artf_ubuntu_repo="${artf_url}/ubuntu"
sudo sed -i -e "/xenial-security/ s|http://security.ubuntu.com/ubuntu|${artf_ubuntu_repo}|g" /etc/apt/sources.list
sudo sed -i -e "s|http://us.archive.ubuntu.com/ubuntu|${artf_ubuntu_repo}|g" /etc/apt/sources.list
echo "deb ${artf_custom_repo} xenial main" | sudo tee -a /etc/apt/sources.list
# revert if needed
#sudo sed -i -e "/xenial-security/ s|${artf_ubuntu_repo}|http://security.ubuntu.com/ubuntu|g" /etc/apt/sources.list
#sudo sed -i -e "s|${artf_ubuntu_repo}|http://us.archive.ubuntu.com/ubuntu|g" /etc/apt/sources.list
#sudo sed -i -e "/ubuntu-packages/d" /etc/apt/sources.list
sudo apt install iproute2 ethtool i40e-dkms

Error:

$ apt changelog iproute2 ethtool i40e-dkms
E: Failed to fetch changelog:/iproute2.changelog  Changelog unavailable for iproute2=4.9.0-1ubuntu2.1~ubuntu16.04.1
E: Failed to fetch changelog:/ethtool.changelog  Changelog unavailable for ethtool=1:4.8-1~ubuntu16.04.1
E: Failed to fetch changelog:/i40e-dkms.changelog  Changelog unavailable for i40e-dkms=2.4.6-0
$

Because of how Jenkins builds images resulting from UCP code development, the Gerrithub project needs to be Fast-Forward Only. Otherwise CICD is publishing dirty images with missing code since Rebase If Necessary rebases on merge and the images are being built pre-merge, thus pre-rebase.

Would it be possible to have Jenkins include the full image path when it posts a Gerrithub comment noting a successful run of the images job? Something like the below

Patch Set 3: Verified+1

Build Successful 

http://12.37.173.196:32775/repository/att-comdev-jenkins-logs/Codereview/att-comdev/drydock/405907/att-comdev/drydock-405907-6491.log : SUCCESS

http://10.24.20.18:30081/job/images/job/att-comdev/job/drydock/job/drydock/358/ : SUCCESS

Image artifact: artifacts-aic.atlantafoundry.com/att-comdev/drydock:29f416aefed900d022dd072a5563f140a5a54229

The “latest” tag doesn’t seem to be tracking with the latest image in artifactory for pegleg : https://artifacts-aic.atlantafoundry.com/artifactory/webapp/#/artifacts/browse/tree/General/artifacts-aic/att-comdev/pegleg

This is an issue is because a script referenced in treasuremap points to latest and instructions would not indicate to acquire a different image.

I have a patchset that moves that’s script’s image target from quay.io to artifactory (since there doesn’t seem to be replication of pegleg to quay.io)
https://review.gerrithub.io/#/c/404208/

I think two possible resolutions here are:
(1) replicate the latest merge build to quay.io and we remain with the quay.io image reference (and abandon my patchset)
(2) update so the latest tag is applied in artifactory to the latest merge build (and accept my patchset)

Put a try/catch around node deletion that will set the build status to SUCCESS if it was previously in a SUCCESS state. Looks like sometimes the deletion of nodes is taking longer than expected and fails the build.

Automodules used for documentation are not being displayed correctly in ReadTheDocs. You can see examples 0 and 1.
Per 2, it is believed that the admin needs to enable the virtualenv in the settings.

The documentation defining UCP development standards from github.com/att-comdev/ucp-integration should be published to readthedocs.io w/ other UCP documentation for easy reference.

Can we retrigger a pipeline if the Virtual Machines fail to provision?

It appears that there is either a delay or failure to replicate a merged commmit from Gerrithub to github.

https://review.gerrithub.io/#/c/402182/
merged 5:11AM 3/2/2018
not showing in:
https://github.com/att-comdev/deckhand/commits/master
as of 8:21AM

It does appear that the merge happened before the build was complete? (5:11 vs. 5:23) - maybe this has something to do with it?