An indirect dependency failed to build due to gcc updates in Tumblweed. This was addressed against:
Issue: rockstor/rockstor-core#2543
PR: rockstor/rockstor-core#2544
Post the fix being merged we need to update our changelog accordingly.

We have a couple of typos in our current README.md

• "git checkout testing # or stable, or your issue branch" - 'stable' references the associated release: this should be the branch name 'master'.
• "... is to ignoe gpg checks ..." - 'ignore'.
• "zypper --no-gpg-checks /usr/src/packages/RPMS/x86_64/rockstor-4.5.7-0.x86_64.rpm" - is missing the 'in' or 'install' command after '--no-gpg-checks'.

We have a long standing limitation in rockstor-core (issue reference pending) regarding a single-issue/entry limitation in our spec file 'changelog' section. If we have, as has just occured in a preliminary 5.0.6-0 release, the following changelog entry:

-Update Poetry build system & normalise on Python 3.11 #2703 #2754 #2693 @phillxnet @FroggyFlox 

Existing installs are unable to parse the multiple GitHub issue references contained in what are future rpm package changelogs to them:

With the resulting auto-generated by Web-UI GitHub 'link' covering all 3 issues (as in picture above) and consisting of:

https://github.com/rockstor/rockstor-core/issues/2703%20#2754%20#2693

Which does not relate to a single issue and results in non being displayed.

For this 5.0.6-0 release it is proposed that delete this current release, resolve a fix in this repo only, then move to re-release 5.0.6-0 here to match those in core & jslibs.

We have a new GitHub workflow file now in master that is required to also be in our testing branch. Resolve by cherry picking or the like.

Follow-up to issue #18, and pr #19:
Add GitHub workflow to edit spec file post rockstor-core release #18 #19
relating to the master branch in this repo.

Umbrella issue for ongoing changelog development associated with the next rpm release.
This issue should only be 'Fixed' by the final changelog & Version/jslibs_version change pull request.
There-after we tag this repo with the subjects version.

As per partner issues in rockstor-core:
rockstor/rockstor-core#2564
rockstor/rockstor-core#2620

We need to update our rpm dependencies in the testing branch as follows:

python-devel -> python3-devel

with an implied (dependant) change that we update to Python major version:

python -> python3

"python3" Implies Python 3.6.15 as the 'default' in Leap 15.3/15.4/15.5

However there is no such package any-longer in Tumbeweed!
So we use the nearest place-holder for development of Py3.8

This issue is connected to rockstor/rockstor-core#2596.

Upon each rockstor/rockstor-core release, we need to edit rockstor.spec accordingly. This process is done manually and thus very time-consuming.

It is proposed that we partly automate this process by preparing the required changes automatically every time a release in rockstor/rockstor-core is made.

Found a couple of cosmetic adjustments.

Also, I am not clear on this statement:
...
The prior installs rockstor-pre.service (initrock) would have asserted these conditions.
...

Thanks to @FroggyFlox & forum members agjbond007 & anborn; for helping to identify this issue. It is currently supposed that, as a result of zypper dup OS version transitions, given examples were (15.3->15.4) & (15.2->15.3->15.4) the, required for Poetry venv build, pg_config file can become unrepresented: i.e. no longer having an alternatives link. This file, as installed by our dependence on:

postgresql13-server-devel

Is normally available via:

rleap15-5:~ # rpm -q --whatprovides /usr/bin/pg_config
postgresql13-server-devel-13.10-150200.5.37.1.x86_64

rleap15-4:~ # ls -lah /usr/bin/pg_config
lrwxrwxrwx 1 root root 27 Nov 14  2022 /usr/bin/pg_config -> /etc/alternatives/pg_config
rleap15-4:~ # ls -lah /etc/alternatives/pg_config
lrwxrwxrwx 1 root root 35 Nov 14  2022 /etc/alternatives/pg_config -> /usr/lib/postgresql13/bin/pg_config
rleap15-4:~ # update-alternatives --list postgresql
/usr/lib/postgresql13

However, during a zypper dup, the new default postgresql version of 14 is installed as per agjbond007 report in the linked forum thread:

i  | postgresql                            | Basic Clients and Utilities for PostgreSQL                                                                                                                                                            | package
i  | postgresql-devel                      | PostgreSQL development header files and libraries                                                                                                                                                     | package
i  | postgresql-llvmjit                    | Just-in-time compilation support for PostgreSQL                                                                                                                                                       | package
i  | postgresql-llvmjit-devel              | Helper package to pull all dependencies to build with llvm support                                                                                                                                    | package
i  | postgresql-server                     | The Programs Needed to Create and Run a PostgreSQL Server                                                                                                                                             | package
i  | postgresql-server-devel               | PostgreSQL server development header files and utilities                                                                                                                                              | package
i  | postgresql13                          | Basic Clients and Utilities for PostgreSQL                                                                                                                                                            | package
i  | postgresql13-devel                    | PostgreSQL client development header files and libraries                                                                                                                                              | package
i  | postgresql13-llvmjit                  | Just-in-time compilation support for PostgreSQL                                                                                                                                                       | package
i  | postgresql13-llvmjit-devel            | PostgreSQL development files for extensions with LLVM support                                                                                                                                         | package
i  | postgresql13-server                   | The Programs Needed to Create and Run a PostgreSQL Server                                                                                                                                             | package
i  | postgresql13-server-devel             | PostgreSQL server development header files and utilities                                                                                                                                              | package
i  | postgresql14                          | Basic Clients and Utilities for PostgreSQL                                                                                                                                                            | package
i  | postgresql14-llvmjit                  | Just-in-time compilation support for PostgreSQL                                                                                                                                                       | package
i  | postgresql14-server                   | The Programs Needed to Create and Run a PostgreSQL Server 

But in the above we see that there is no counterpart to the postgresql13-server-devel installed in the 14 variant, and thus no counterpart provider (via alternatives) of our required pg_config - used by our Poetry dependencies manager to build our Python dependencies upon install/update.

The fix provided by @FroggyFlox in the linked forum thread is to re-assert our dependency upon postgresql13 (and it's server-devel counterpart) via a move away from the auto alternative (as higher priority due to newer) which has unfortunately obscured (removed from alternative managed path) the requried for venv build 'pg_config' utility.

The following shows the by-hand intervention currently required for installs that have previously been 'zypper dup' (distribution updated) from a prior Leap version:

alternatives --config postgresql
There are 2 choices for the alternative postgresql (providing /usr/lib/postgresql).

  Selection    Path                   Priority   Status
------------------------------------------------------------
* 0            /usr/lib/postgresql14   140       auto mode
  1            /usr/lib/postgresql13   130       manual mode
  2            /usr/lib/postgresql14   140       manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/lib/postgresql13 to provide /usr/lib/postgresql (postgresql) in manual mode

alternatives --config postgresql
There are 2 choices for the alternative postgresql (providing /usr/lib/postgresql).

  Selection    Path                   Priority   Status
------------------------------------------------------------
  0            /usr/lib/postgresql14   140       auto mode
* 1            /usr/lib/postgresql13   130       manual mode
  2            /usr/lib/postgresql14   140       manual mode

Press <enter> to keep the current choice[*], or type selection number: 

Proposal

We currently have an rpm defined dependency on postresql13 and it's associated server and server-devel packages so rather than move all these dependency over the the newer, OS default, of 14, we should assert our system requirement via the alternative system. At least until we update our own client library of (from pyproject.toml):

psycopg2 = "==2.8.6" # last Python 2.7 version, PostgreSQL 13 errorcodes map?

This older dependency, that is questionable re Postgresql 13, should likely not be tried further with postgresql 14. In the context of our master and testing branches currently span Python 2.7 / 3.6.

This proposed postgresql alternatives assertion could be accomplished within a scriptlet, prior to our Poetry venv build, and post our new rpm/scriptlets install (which would have already installed our dependencies): e.g. %post.

Referenced:

• SuSe SLE: https://documentation.suse.com/sles/15-SP5/html/SLES-all/cha-update-alternative.html

• Forum threads:
  -- agjbond007 - https://forum.rockstor.com/t/upgrade-to-5-0-1-0-from-4-6-1-rockstor-fails/8905
  -- anborn - https://forum.rockstor.com/t/system-unavailable-after-update-to-stable-4-6-1-0-release/8904

Follow-up to #17 & #26 but for Python 3.9 across all our targets. RPM spec file counterpart to rockstor/rockstor-core#2691 as we need to modify our dependencies to ensure our all targets move to Py3.9 as a dependency.

Issue to associated with ongoing 4.5.8-0 changelog development. We are nearing this next release so we need to begin testing the resulting changelog entries. This issue should only be 'Fixed' by the final move to 4.5.8-0 both within the changelog and the then to-be matching 'Version' & 'jslibs_version' entries. Where-up we also tag this repo accordingly upon an rpm being release with the tags indicated version, likely 4.5.8-0 at that time.

Following our recent accommodation re venv python updating in-place via #31 brute force wipe, we are cutting our own python/django legs off. This leads to a requirement to command line reboot. We likely have refinement regarding our systemd service handling that can be approached to improve this user experience.

See:

Thanks to @FroggyFlox and forum members Mark93 & eriklysoe we have a known failure re rpm update within our venv rebuild when we change our Python version. First, and currently most recently observed in rpm version-release 5.0.0-0.

Poetry, our dependency manager, has to date functioned flawlessly (even thought we use an early/old version) but we look now to have run into a limitation (old bug). During uninstall we remove .venv but during update we do not - intentionally. This enables re-use of existing unchanged, and so pre-build in a prior update/install, dependencies within our venv. However when we update our Python version, as was done fist in 5.0.0-0 (first release in our new testing phase) the required complete rebuild of the .venv, against the newly updted Python version, did not take place.

From @FroggyFlox's forum exposition here:
https://forum.rockstor.com/t/rockstor-pre-service-failing-after-update-to-5-0-0-0/8899/10
we have the following contents from our diagnostic file: poetry-install.txt, used to capture the output of our %posttrans scriptlet run of build.sh; which in turn invokes our run/rerun of a Poetry install - functioning as our venv maintenance re updating our venv:

cat poetry-install.txt 
Installing dependencies from lock file

  SolverProblemError

  The current project's Python requirement (2.7.18) is not compatible with some of the required packages Python requirement:
    - distro requires Python >=3.6, so it will not be satisfied for Python 2.7.18
  
  Because distro (1.8.0) requires Python >=3.6
   and no versions of distro match !=1.8.0, distro is forbidden.
  So, because rockstor depends on distro (*), version solving failed.

  at ~/.local/share/pypoetry/venv/lib64/python3.6/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
    
    For distro, a possible solution would be to set the `python` property to "<empty>"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

Since "Move build.sh execution from %posttrans to rockstor-build.service #61" #63 the rpmbuild scriptlets no longer manage the installation of Poetry:

Removal of redundant Poetry install/update. We now assume this of our build host, and assert rockstor-build.service / build.sh as the sole source of truth for this.

resulting in %build% failing with poetry: command not found when attempting to run rpmbuild on non prior rockstor instances: i.e. where rockstor-build.service (build.sh) has already arranged our poetry dependency via our preferred python311-pipx arrangement.

Ideally we need an additional README.md section detailing to the specific build host requirements assumed. We currently have the following paragraph:

Using an existing Rockstor install ensures that at least our prior dependencies are in place. This helps with establishing if any new dependencies are required when making spec file changes. The rpmbuild process, within the %check scriptlet, also runs all our existing unit tests. As such a properly configured and running Postgres server is required. The prior installs rockstor-pre.service (initrock) would have asserted these conditions. Otherwise the rpmbuild command will fail on the %check scriptlet stage.

It is proposed that this be moved to for example a Build Host assumptions with details on how, using the rockstor-core code (relevant branch) where possible, can be sourced to effect the buildhost requirements for rpmbuild. This would also assist in ensuring the minimal changes required to rpmbuild on for example a MinimalVM openSUSE image derived build host.

Regarding in-development changes in rockstor-core re: Adopt dedicated secrets management library in current draft status PR: rockstor/rockstor-core#2756 we have a new OS dependency to provide secrets storage. The proposal is as follows:

password-store: https://www.passwordstore.org/

Authored/maintained by Jason A. Donenfeld of wireguard fame.

Requiring a new dependency across all targets, and likely some rpmbuild spec file scriptlet additions (as yet unclear).

Our official downloadable installer, itself built using the https://github.com/rockstor/rockstor-installer repo, has all rpm build prerequisites preinstalled; bar currently wget, to be addressed in rockstor/rockstor-installer#133 "Proposal to add wget to our included packages".

It is proposed that we base our README.md contents around in-place rpm rebuilds to enable complete end-to-end testing if code changes in any relevant issue also require modifications in this repo. We also then have an easier path for domain experts to contribute improvements in this our initial Poetry build approach.

We should also mention in this repo-heading documentation that we mirror the master/testing branch structure of rockstor-core for, respectively the Stable and Testing channel updates. We are however on RC4 currently so will initially only have a testing branch in this repo with any significant contents.

We have likely already stretched our use of the rpm packaging systems %posttrans for our venv, initial secrets setup, and jslibs preparation, via the build.sh rockstor-core bash script. Move the execution of build.sh from outside of the constraints of the packaging system, to the level of OS orchestration: systemd.

Partner issue to:

• rockstor-core's: "Add rockstor-build systemd service": rockstor/rockstor-core#2793
• rockstor-installer: "Tumbleweed: build.sh fails as no /etc/locale.conf": rockstor/rockstor-installer#156

Partner PR:

• rockstor-core's: "Add rockstor-build systemd service ...": rockstor/rockstor-core#2795

Counterpart to rockstor-core issue by the same title:
rockstor/rockstor-core#2529
See referenced rockstor-core issue for context.

In essence we require matched branches for our build system to maintain separation between changes required for our stable channel maintenance (master branch to come) and our testing channel for more major development (testing branch ongoing).

As a partial solution to rockstor/rockstor-core#2639 we could/should:

1. Have incomming outgoing package scriptlet (removed or upgraded) copy it's ´/opt/rockstor/static/config-backups´ contents to top-level ´/opt/rockstor/config-backups´ directory.
2. Have incoming package scriptlet (installed or upgrading) copy what-ever it finds in the ´/opt/rockstor/config-backups´ directory to the working location accessible via Web-UI: and in doing so re-establish what the existing db expects to find there.

A bonus of this process is that on every package managed upgrade we make a copy of our config backup files.

This proposed ´/opt/rockstor/config-backups´ directory would not be deleted in any package clean-up process. And so would serve as a repository for such files, while still maintaining our isolation form the OS at large and keeping within our parent directory.

We make use of many servcie macros to administer our systemd services. Clarify via comments to assist in maintenance.

Older installs resulting from Rockstor-Leap15.3-generic.x86_64-4.1.0-0.install.iso Jan 14 2022 or earlier, that have also been zypper duped to at least a 15.4 base OS will now require a DB update prior to being compatible with the Django LTS 4.2 & psycopg3 that are used in current testing (5.0.6-0) and proposed next stable of 5.1.0-0.

Partner issue to rockstor-core:
"Establish Postgres database format upgrade": rockstor/rockstor-core#2780

• Establish new dependency to make pg_upgrade for target Postgres (13) available.
• Run via install/update scriptlets the shell script to be made available via the linked partner issue in rockstor-core.

During the RPM %build & %check scriptlets we instantiation and use a Poetry install. Ensure the Poerty version used is in-step with changes implemented in rocksot-core against the following issue:
"Update Poetry build system": rockstor/rockstor-core#2703

See the above linked issue for caveates relating to our current use of the now out-dated 1.1.x series of Poetry.

We currently make almost no use of the BuildRequires section. It is proposed that we bring this key rpmbuild feature into play more to enable a better out-of-the-box experience re rpmbuild runs on say Minimal-VM images. Our new Poetry based install method requires that we build many of our required dependencies, and so we have specified these within the final install dependencies section. But this ignores the initial rpmbuild requirements that can be well served by BuildRequires.

These changes should be address first in the testing branch.

We appear to be failing to stop our outgoing systemd services during an upgrade. This may be down to a legacy forced/premature removal of these service files prior to later automated mechanisms intended to address this.

This issue is a spin-off from:

rockstor/rockstor-core#2651