Comments (4)
dennis-behm
commented on September 16, 2024
Hi @FALLAI-Denis ,
The zAppBuild property applicationSrcDirs is meant to limit the scope to a subset of the directories in your repository.
So, when running it for
applicationSrcDirs=${application}/cobol,${application}/copybook
It indeed stores the hash twice, but limits it to these two directories.
*** Obtaining hash for directory /u/dbehm/test-zapp/dbb-zappbuild/samples/MortgageApplication/cobol
** Setting property :githash:MortgageApplication/cobol : 4b4dafa2bb71094cd7cdf2a3a2872e9416f81807
** Setting property :giturl:MortgageApplication/cobol : https://github.com/dennis-behm/dbb-zappbuild.git
*** Obtaining hash for directory /u/dbehm/test-zapp/dbb-zappbuild/samples/MortgageApplication/copybook
** Setting property :githash:MortgageApplication/copybook : 4b4dafa2bb71094cd7cdf2a3a2872e9416f81807
** Setting property :giturl:MortgageApplication/copybook : https://github.com/dennis-behm/dbb-zappbuild.git
** Writing build report data to /u/dbehm/test-zapp-app-out/mortgageout/build.20221111.022013.020/BuildReport.json
** Writing build report to /u/dbehm/test-zapp-app-out/mortgageout/build.20221111.022013.020/BuildReport.html
** Updating build result BuildGroup:MortgageApplication-242_preMerge BuildLabel:build.20221111.022013.020
** Build ended at Fri Nov 11 14:20:34 GMT+01:00 2022
Also impactBuild is working with it. What type of issue do you see there?
When you would like to manage multiple git repositories as the build scope, zAppBuid can manage that. Please have a look to the publication about build scopes at https://www.ibm.com/support/pages/node/6381788
But zAppBuild does not magically understand all git repos under the specified root directory for --workspace. This is configured by the applicationSrcDirs. So, one option might be to parse your .imports configuration - where you have specified the different included repos. Btw - is this .imports your own invention or follows a particular standard?
Dennis
from dbb-zappbuild.
FALLAI-Denis
commented on September 16, 2024
Hi @dennis-behm
What type of issue do you see there?
We create a manifest file (in part) from the BuildReport.json file, which must correspond to the contents of the components package taken from the main git repository (consolidation of impactsBuilds to produce a complete package)..
Due to the duplicates on :giturl: and :githash: this caused us several problems:
- in the manifest file we want to indicate the reference of the main repository (the one that contains the programs), and we don't need the information on the secondary repositories (the ones that contain the copybooks), but we still need to have a way to distinguish them
- if you indicate several folders in the main repository, to filter the content of the main repository, you end up with as many
:giturl:and:githash:, all with the same value, and you must only take one of them consideration
applicationSrcDirs=${application}/src,${application}/pacbase,${application}/.imports/ENVIRONN,${application}/.imports/GCL
{
"id": "DBB.BuildResultProperties",
"type": "PROPERTIES",
"properties": {
":githash:ceab3-environn-outinfr-central-test/.imports/GCL": "da96cca4dc8ae0325f4b6f46257d646bb15256ca",
"filesProcessed": "7",
":giturl:ceab3-environn-outinfr-central-test/pacbase": "https://xxxxxx/scm/CEH49-CENTRAL-ENVIRONN-OUTINFR/ceab3-environn-outinfr-central-test.git",
"buildResult": "CLEAN",
":githash:ceab3-environn-outinfr-central-test/.imports/ENVIRONN": "c4ef1816f05ddf08eab6ef5a4ec0dd054c6fc758",
":giturl:ceab3-environn-outinfr-central-test/.imports/ENVIRONN": "https://xxxxxx/scm/ceh49-central-environn-outinfr/ceab3-environn-outinfr-central-test-ressources.git",
":githash:ceab3-environn-outinfr-central-test/src": "f2c3f2dcf613709802c2b3a4c932c3b2f681533f",
":giturl:ceab3-environn-outinfr-central-test/.imports/GCL": "https://xxxxxx/scm/cegcl-central-gcl-outinfr/ceq66-gcl-outinfr-central-ressources.git",
"fullBuild": "true",
":githash:ceab3-environn-outinfr-central-test/pacbase": "f2c3f2dcf613709802c2b3a4c932c3b2f681533f",
":giturl:ceab3-environn-outinfr-central-test/src": "https://xxxxxx/scm/CEH49-CENTRAL-ENVIRONN-OUTINFR/ceab3-environn-outinfr-central-test.git"
}
}
We have as many :giturl: and :githash: for the main repository, ceab3-environn-outinfr-central-test.git, as there are folders referenced in applicationSrcDirs, here twice.
Example manifest file to produce (here from a fullBuild): note that the COMMIT information, line type 15, is false because we did not select the correct :githash: (we took that of a secondary repository of COPYBOOKs and not that of the main repository).
11 GIT PROJET CEH49-CENTRAL-ENVIRONN-OUTINFR
12 GIT DEPOT ceab3-environn-outinfr-central-test
13 GIT BRANCHE master
14 GIT SERVER https://xxxxxxxx/scm/CEH49-CENTRAL-ENVIRONN-OUTINFR/ceab3-environn-outinfr-central-test.git
15 GIT COMMIT https://xxxxxxxx/projects/CEH49-CENTRAL-ENVIRONN-OUTINFR/repos/ceab3-environn-outinfr-central-test/commits/da96cca4dc8ae0325f4b6f46257d646bb15256ca
21 BUILD ID build.20221114.110821.008
22 BUILD URL https://yyyyyyyy/dbb/rest/buildResult/12072
23 BUILD SOURCES 00000007
24 BUILD TYPE fullBuild
89 SYSTEM TYPE CODE PATH HORODATAGEBUILD TYPE_SRC CODE_SRC CHANGE RC_COMPIL HORODATAGE_SRC PROCESSOR
89 X(8) X(8) X(8) X(44) X(15) X(10) X(8) X(8) X(10) X(15) X(8)
90 SIRIS DBRM S9TST1 SIRIS/DBRM 20221114-110821 COBOL S9TST1 T00001 0004 20221110-144328 COBSTD00
90 SIRIS LOADBT S9TST1 SIRIS/LOADBT 20221114-110821 COBOL S9TST1 T00001 0004 20221110-144328 COBSTD00
90 SIRIS DBRM S9TST2 SIRIS/DBRM 20221114-110821 COBOL S9TST2 T00001 0004 20221110-181829 COBSTD00
90 SIRIS LOADBT S9TST2 SIRIS/LOADBT 20221114-110821 COBOL S9TST2 T00001 0004 20221110-181829 COBSTD00
90 SIRIS DBRM S9TST3 SIRIS/DBRM 20221114-110821 COBOL S9TST3 T00001 0004 20221003-103009 COBSTD00
90 SIRIS LOADBT S9TST3 SIRIS/LOADBT 20221114-110821 COBOL S9TST3 T00001 0004 20221003-103009 COBSTD00
90 SIRIS DBRM S9TST5 SIRIS/DBRM 20221114-110821 COBOL S9TST5 T00001 0004 20220915-170333 COBSTD00
90 SIRIS LOADBT S9TST5 SIRIS/LOADBT 20221114-110821 COBOL S9TST5 T00001 0004 20220915-170333 COBSTD00
90 HUB DBRM S9TSTA HUB/DBRM 20221114-110821 COBOL S9TSTA T00001 0004 20220909-170851 COBSTD00
90 HUB LOADBT S9TSTA HUB/LOADBT 20221114-110821 COBOL S9TSTA T00001 0004 20220909-170851 COBSTD00
90 HUB DBRM S9TSTB HUB/DBRM 20221114-110821 COBOL S9TSTB T00001 0004 20221003-103009 COBSTD00
90 HUB LOADBT S9TSTB HUB/LOADBT 20221114-110821 COBOL S9TSTB T00001 0004 20221003-103009 COBSTD00
90 HUB DBRM S9TSTC HUB/DBRM 20221114-110821 COBOL S9TSTC T00001 0004 20221003-103009 COBSTD00
90 HUB LOADBT S9TSTC HUB/LOADBT 20221114-110821 COBOL S9TSTC T00001 0004 20221003-103009 COBSTD00
99 SIRIS DBRM 00000004
99 SIRIS LOADBT 00000004
99 HUB DBRM 00000003
99 HUB LOADBT 00000003
So, one option might be to parse your .imports configuration - where you have specified the different included repos. Btw - is this .imports your own invention or follows a particular standard?
This is a proprietary solution that we have implemented but looks a lot like a solution described in Managing the build scope in IBM DBB builds with IBM zAppBuild.
The difference is that we don't want to carry the import information through the Jenkinsfile for the following reasons:
- the people of the development project team (of which I am not a member) are responsible for declaring the dependencies between the main repository of the programs and the secondary repositories of the copybooks: they do not have to intervene on the technical configuration of the constructions , especially on the Jenkinsfile.
- the people of the tools and procedures administration team (of which I am a member) are responsible for the Jenkins pipeline and the implementation of IBM Dependency Based Build: they do not have to manage the declarations of dependencies between the programs and copybooks
Our Jenkinsfile is reduced to its simplest form (and everything is managed in a Sharedlibrary):
@Library('LibrairieAtelierCentral') _
buildCentral(mode:"dbb", typeBuild:'impactBuild');
We can switch from an impactBuild to a fullBuild and vice versa by modifying the typeBuild parameter of the buildCentral function.
from dbb-zappbuild.
FALLAI-Denis
commented on September 16, 2024
Hi,
Could using the --impactScope as described in the Wiki, Use Whitelists in impactBuild scenario, be an alternative to listing the folders to consider rather than the root of the main Git repository?
Is the --impactScope, or something equivalent, usable for a --fullBuild? Even generalized to all build modes?
From which versions of zAppbuild and IBM DBB is this feature available?
Since the organization of our Git repositories is standardized and they all use the same folder organization, can the impactScopeList property be set at the server configuration level?
That said, a quick analysis of the proposed solution shows that the filtering by the scope occurs after the construction of the buildlist...
It seems to me preferable that the filtering by the scope intervenes at the time of the construction of the buildlist, by filtering the folders / files to be analyzed rather than by eliminating only a posteriori to remove elements from the buildlist.
Perhaps this is a solution that we can learn from to filter upstream rather than downstream of the construction of the buildlist.
Thanks.
from dbb-zappbuild.
FALLAI-Denis
commented on September 16, 2024
Hi,
I return to the problems posed by the declaration of the applicationSrcDirs property.
To date, this property is used by zAppbuild to do two things which in my opinion should be independent:
- used to declare the folders (and dependent subfolders) that must be scanned to detect the sources to be built ; whether in fullBuild mode or in impactBuild mode ; e.g. COBOL sources
- used to declare folders (and dependent subfolders) that should be scanned for changed items that may trigger the construction of other items, but which themselves did not need to be built ; only in impactBuild mode ; e.g. COPYBOOK sources
Knowing that apparently zAppbuild expects that each folder listed in applicationSrcDirs is the root of a Git Repository, which does not correspond to our usage since we are listing folders inside the Git Repository (subfolders in the Git Repository), which has the consequence of generating as many :giturl: and :githash: information as folders, even though they are all in the same Git Repository.
Note: if the impacting sources are not identified in the applicationSrcDirs property, then the impacts are not detected (the impactSearch property is not sufficient to declare the impacting sources, when we thought it was its role).
From my point of view, these two needs should be handled by two independent properties:
applicationSrcDirsproperty: list of folders to analyze for the search for sources to build, e.g. COBOL sources ; to use in fullBuild and impactBuild modeimpactingSrcDirs(or other name) property: list of folders to analyze to find sources that may impact other sources, but which themselves are not to be built, e.g. COPYBOOK sources ; to be used only in impactBuild mode
To see if for each of these folders list it would not also be necessary to manage lists of file extensions to be taken into account (positive list) or to be ignored (negative list).
from dbb-zappbuild.
Related Issues (20)
- Enhance processing of IMS programs HOT 8
- Question : zUnit - While Running the test case - Failed to load the User program getting SOC4 Abend HOT 6
- Reporting capabilities depend on metadata store connection / git repo in workspace HOT 1
- Encoding issue with new OR concatenation HOT 1
- Execute the build on a ZIIP.
- How to delete unwanted feature branch's collection from DB2
- Config JAVA environment settings for ZOS connect V2 HOT 2
- Build support for Java language in zAppbuild HOT 6
- Displaying a summary of build errors in build summary HOT 1
- Reporting capability creates empty files HOT 1
- Housekeeping of unused / dead code
- Issue in DBB User Build after enabling cobol_storeSSI HOT 3
- Line continuation for IDENTIFY statement HOT 1
- Misleading message in verbose output during lookup of last successful build result HOT 1
- Document if changed files got excluded from build scope HOT 1
- Missing properties HOT 1
- DBRM deployType should be driven by properties
- Use filters and folder names in the languageConfigurationMapping.properties HOT 2
- Processing of Binder control statement members
- BuildUtilities.copySourceFiles(BuildUtilities.groovy:100) - EDC5003I Truncation of a record occurred during an I/O operation.; errno=3 errno2=0xc0400060 last_op=60 errorCode=0x5620062 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbb-zappbuild.