ros2 / ros_buildfarm_config Goto Github PK
View Code? Open in Web Editor NEWThis project forked from ros-infrastructure/ros_buildfarm_config
This project forked from ros-infrastructure/ros_buildfarm_config
marti_common was added to the blacklist for Humble and Rolling because of build failures (swri-robotics/marti_common#665). We've updated for Humble, and it should be working now. Could we get removed from the blacklist because we've got downstream users wanting to install this package in Humble.
This is a tracking issue for the investigation in CI build performance.
MAKEFLAGS
to allow further parallelization within a project's build. Results showed that -j1
this was not substantially contributing to the performance shortcomings.We recently added Foxy to our buildfarm. All ROS distros build their SRC and BIN jobs correctly, except for Foxy (our first ROS2 distro in the farm). When jobs are updated from the reconfigure jobs, the ssh-agent credentials appear to not be set properly. This leads to an immediate failure of the job:
java.io.IOException: [ssh-agent] Could not find specified credentials
If I edit the job configuration, the proper credentials are auto-filled. Just opening and saving the job again, lets the jobs run properly.
I'm not sure if the config is the right place for this issue, or if it's another repo.
The Jenkins job priorities have previously been allocated such that certain job types are prioritized above other job types, and within each job type, more recent ROS distributions are prioritized above older ones.
We need to increase the spacing between these job types to avoid priority collisions when we add jobs for Foxy.
Additionally, we should consider adding a group for the manual CI jobs (overlay) which would be prioritized above the nightly jobs.
Current allocations:
45
jenkins_pull_request_job_priority::eloquent/source-build.yaml
46
jenkins_pull_request_job_priority::dashing/source-build.yaml
47
jenkins_pull_request_job_priority::crystal/source-build.yaml
48
jenkins_job_priority::eloquent/ci-nightly-connext.yaml
jenkins_job_priority::eloquent/ci-nightly-cyclonedds.yaml
jenkins_job_priority::eloquent/ci-nightly-debug.yaml
jenkins_job_priority::eloquent/ci-nightly-extra-rmw-release.yaml
jenkins_job_priority::eloquent/ci-nightly-fastrtps-dynamic.yaml
jenkins_job_priority::eloquent/ci-nightly-fastrtps.yaml
jenkins_job_priority::eloquent/ci-nightly-navigation2-prerequisites.yaml
jenkins_job_priority::eloquent/ci-nightly-navigation2.yaml
jenkins_job_priority::eloquent/ci-nightly-opensplice.yaml
jenkins_job_priority::eloquent/ci-nightly-release.yaml
jenkins_job_priority::eloquent/ci-overlay.yaml
49
jenkins_job_priority::dashing/ci-nightly-connext.yaml
jenkins_job_priority::dashing/ci-nightly-cyclonedds.yaml
jenkins_job_priority::dashing/ci-nightly-debug.yaml
jenkins_job_priority::dashing/ci-nightly-extra-rmw-release.yaml
jenkins_job_priority::dashing/ci-nightly-fastrtps-dynamic.yaml
jenkins_job_priority::dashing/ci-nightly-fastrtps.yaml
jenkins_job_priority::dashing/ci-nightly-opensplice.yaml
jenkins_job_priority::dashing/ci-nightly-release.yaml
jenkins_job_priority::dashing/ci-overlay.yaml
50
jenkins_job_priority::crystal/ci-nightly.yaml
jenkins_job_priority::crystal/ci-overlay.yaml
55
jenkins_commit_job_priority::eloquent/source-build.yaml
56
jenkins_commit_job_priority::dashing/source-build.yaml
57
jenkins_commit_job_priority::crystal/source-build.yaml
75
jenkins_source_job_priority::eloquent/release-bionic-arm64-build.yaml
jenkins_source_job_priority::eloquent/release-bionic-armhf-build.yaml
jenkins_source_job_priority::eloquent/release-build.yaml
76
jenkins_source_job_priority::dashing/release-bionic-arm64-build.yaml
jenkins_source_job_priority::dashing/release-bionic-armhf-build.yaml
jenkins_source_job_priority::dashing/release-build.yaml
77
jenkins_source_job_priority::crystal/release-bionic-arm64-build.yaml
jenkins_source_job_priority::crystal/release-build.yaml
79
jenkins_binary_job_priority::eloquent/release-bionic-arm64-build.yaml
jenkins_binary_job_priority::eloquent/release-build.yaml
80
jenkins_binary_job_priority::dashing/release-bionic-arm64-build.yaml
jenkins_binary_job_priority::dashing/release-build.yaml
81
jenkins_binary_job_priority::crystal/release-bionic-arm64-build.yaml
jenkins_binary_job_priority::crystal/release-build.yaml
99
jenkins_binary_job_priority::dashing/release-bionic-armhf-build.yaml
jenkins_binary_job_priority::eloquent/release-bionic-armhf-build.yaml
This is a subset of the greater Rolling transition to 22.04 ros2/ros2#1221
rolling_rosdistro-cache
rolling_failing-jobs
Rci_reconfigure*
Rdev_reconfigure*
Rdev_trigger*
Rdoc_reconfigure*
Rdoc_trigger*
Rrel_*
Rsrc_uF__*
Rbin_uF64__*
Rbin_ufv8_uFv8__*
Rdev__*
Rpr__*
Rdoc__*
Since this will be the first time I've updated the platform of an existing rosdistro there are a few things I'm trying to work out to smooth this out going forward.
The target platform and architecture is embedded into all job names, but is only part of the Jenkins view name for release jobs.
Non-default build files have the build file name embedded in the view but default jobs do not. This means that when we bring up the Jammy source (Rdev) jobs they'll create doubled jobs for everything on the build farm or de-list the old focal jobs. I haven't yet checked how the views are updated to check if the old jobs will just remain or be removed. Since we aren't supporting the focal jobs any more the natural thing to do would be to delete them clearing space for the new jammy jobs. However that also means that we lose the build history of those jobs immediately and there is probably value in having a sunset period where the jobs are disabled but not deleted just like the release jobs will be. The same issue true of Rpr, Rci, and Rdoc jobs.
I see a couple of methods to resolve this:
We could update ros_buildfarm to include a target identifier in all views regardless of job type. I've not yet looked to see how feasible this would be to change. Sometimes ros_buildfarm changes are small thanks to templating and code re-use and sometimes the same function is implemented across several different layers or in python and bash or groovy. This change would also affect all distributions (and ROS 1 distros) not just Rolling which may or may not be desirable.
We could add the Jammy jobs by introducing new non-default targets which would put them in new views based on the build file identifier used in the config index. If we did this I'd suggest that for Rolling we abandon the use of default build files for all build types except release build files where the target is encoded in the view already. This has the benefit of requiring no changes in ros_buildfarm but the downside of creating separate conventions for release build files and every other build file as well as for Rolling compared to every other distribution where there is less of an issue using default build files because the platform list is less likely to change during the life of the distribution.
After updating build files and deploying the new jobs we manually move the focal jobs into manually created views which we can clean up at any later date. This is pretty straightforward to do with a Groovy script or Jenkins configuration UI but it's a chore and will have to be maintained separately.
We could do nothing and let the jobs fall where they will, and see how much, if any trouble the extra jobs give us.
We could delete outright the focal jobs with no sunset period in order to clear the way for the Jammy jobs and not change anything else.
I've been thinking of this as heterogeneous build agents since the agents can disparate types of jobs but the actual important part is returning to a homogeneous agent pool. So I put the Latin down and called this issue "unified" build agents.
This rollout is happening in a number of stages:
Background: When we first put the CI jobs on the ROS build farm we ran into the issue that one CI job from ci.ros2.org which was the prototype for them is used to having a full build agent's resources rather than 1/4th of a build agent and we ran into performance issues as a result.
ros-infrastructure/ros_buildfarm#839 added support for the heavy job plugin which allows jobs to assign a "weight" which is the number of executors required to host that job.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.