aosc-dev / acbs Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 8.0 688 KB

A Package Tree Management Utility

Home Page: https://AOSC-Dev.github.io/acbs

License: GNU Lesser General Public License v2.1

Python 94.17% Shell 4.10% C++ 1.38% C 0.34%

autobuild

acbs's Issues

Support trees with categories started not with "core/base/extra" for easier external tree support (linux-kernel-extra, aosc-os-arm-bsps, etc)

root@e531 [ acbs@staging ] # cat /etc/acbs/forest.conf 
[default]
location = /usr/lib/acbs/repo

[linux-kernel-extra]
location = /var/lib/acbs/linux-kernel-extra
root@e531 [ acbs@staging ] # ls /var/lib/acbs/linux-kernel-extra
featured-kernel  machine-kernel  march-kernel  README.md  upcoming-kernel
root@e531 [ acbs@staging ] # ls /var/lib/acbs/linux-kernel-extra/upcoming-kernel/
linux-next
root@e531 [ acbs@staging ] # cat /var/lib/acbs/linux-kernel-extra/upcoming-kernel/linux-next/autobuild/defines 
PKGNAME=linux-next
PKGSEC=kernel
PKGDEP=""
BUILDDEP="bc dracut git"
if [[ "${CROSS:-$ARCH}" = "amd64" ]]; then
    BUILDDEP+=" grub"
fi
PKGDES="linux-next branch, use at your own risk."

ABSTRIP=no
ABELFDEP=no
ABRPMAUTOPROVONLY=yes
root@e531 [ acbs@staging ] # ./acbs-build.py -t linux-kernel-extra linux-next
-----------------------------------------------------------------------------------Welcome to ACBS - 20170104-----------------------------------------------------------------------------------

[CRIT]: Oops! ACBSGeneralError: No valid candidate package found for linux-next.

Naming and organization issues

As a Python project, we should observe several conventions:

lib/acbs_*.py should be acbs/*.py
Excessive usage of classes is not needed. General functions should be put at module level
Class names should be CapitalizedWords. acbs_ prefix is also not needed

full mercurial support request

Checkout specefic commit
Checkout specific tag

Spec-and-define writing wizard

A lot of Modern:tm: Package Managers have this magical init command to help people create an initial package-information file interactively. It might be a good idea to make one for the tree thing we have too.

A wizard should be able to ask for and fill in:

Package name and version
Package section (abbs tree and PKGSEC); with tab completion
Package source type selection (tbl, vcs, dummy)
- Automatic addition of checksum
Package description
PKGDEP and BUILDDEP; with tab completion

I recommend using the readline library in Python. It's the old GNU readline with all the Emacs navigation shortcuts, quite useful for moving around "words" in the DEPS. And it seems to allow custom completers.

User story: Look at me. I am that one user, I am dumb and clumsy, and I can't edit text on a high-latency connection. Seriously, I found myself struggling over something this easy.

It would also be great to have a command for adding or removing DEPs for trial-and-error building. That would require reading the file though.

libarchive fails to load even if library installed

Even with libarchive installed (it's actually required by acbs in AOSC OS), the following code always executes.

acbs/acbs/src_process.py

Lines 146 to 147 in 16e000b

 logging.warning( 

 'Failed to load libarchive library! Fall back to bsdtar!')

suggestion: re-download source on checksum mismatch

Currently there is no handling for sources downloaded with mismatched hash checksum, developer/user/packager will have to remove the file to build again.

What if we add another "attempt-based" routine that attempts to re-download tarball(s) again (for a given amount of time by parameter [?])...

git: clone from branch/commit/tag (-b) in the first place, with depth 1

acbs currently does a full-ish clone and then checks out what it needs. That is inefficient af.

If any reusing of the clone is needed, just do a shallow fetch before checking out to what you need.

acbs probably breaks on versioned PKGDEP ("foo<3 bar_ baz>=1.1")

ab3 depset:

$PKGDEP =~ (^| )$1(_|[<>=]=[^ ]*)?( |$)

meta: architecture-related logic statements not processed by ACBS

Originally reported as AOSC-Dev/aosc-os-abbs#568 by @cthbleachbit .

if [[ "${CROSS:-$ARCH}" != arm* ]]; then
	VER=8u111b14
else
	if [[ "${CROSS:-$ARCH}" = armel ]]; then
		VER=8u101b13
	elif [[ "${CROSS:-$ARCH}" = arm64 ]]; then
		VER=8u111b16
	fi
fi

if [[ "${CROSS:-$ARCH}" = "arm64" ]]; then
	SRCTBL="https://repo.aosc.io/aosc-repacks/aarch64-java/openjdk-$VER.tar.xz"
elif [[ "${CROSS:-$ARCH}" = "armel" ]]; then
	SRCTBL="https://repo.aosc.io/aosc-repacks/aarch32-java/openjdk-$VER.tar.xz"
else
	SRCTBL="https://repo.aosc.io/aosc-repacks/openjdk-$VER.tar.xz"
fi

As found with extra-java/openjdk in https://github.com/AOSC-Dev/aosc-os-abbs, current state of ACBS with reading spec files could not produce the correct logical result - as seen in the example above. It is no longer possible to build a "special" version for armel with ACBS.

This issue is migrated here as suggested by @gumblex .

Incorrect dependency resolve...

Don't know how to explain this but... Here you go...

root:/usr/lib/abbs/repo# acbs-build -d shared-mime-info
---------------------------------------Welcome to ACBS - 20170103---------------------------------------
[DEBUG]: Search Path: /usr/lib/abbs/repo
[INFO]: Start building extra-misc/shared-mime-info
[DEBUG]: Found pm_backends: ['/usr/lib/acbs/acbs/pm/dpkg.sh']
[INFO]: Checking for dependencies, this may take a while...
[DEBUG]: Searching dependencies: ['intltool', 'glib', 'libxml2']
[DEBUG]: PM Backend: dpkg CMD: pm_exists('intltool') Res: False
[DEBUG]: PM Backend: dpkg CMD: pm_exists('glib') Res: False
[DEBUG]: PM Backend: dpkg CMD: pm_exists('libxml2') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_repoquery('glib') Res: False
[DEBUG]: PM Backend: dpkg CMD: pm_repoquery('intltool') Res: False
[INFO]: Building in-tree dependencies: glib intltool
[DEBUG]: New build thread started for glib
[DEBUG]: Search Path: /usr/lib/abbs/repo
[INFO]: Start building base-libs/glib
[DEBUG]: Found pm_backends: ['/usr/lib/acbs/acbs/pm/dpkg.sh']
[INFO]: Checking for dependencies, this may take a while...
[DEBUG]: Searching dependencies: ['python-2', 'docbook-xml', 'gtk-doc', 'libxslt', 'pkg-config', 'pcre', 'libffi', 'elfutils']
[DEBUG]: PM Backend: dpkg CMD: pm_exists('python-2') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('docbook-xml') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('gtk-doc') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('libxslt') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('pkg-config') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('pcre') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('libffi') Res: 
[DEBUG]: PM Backend: dpkg CMD: pm_exists('elfutils') Res: 
[INFO]: All dependencies are met. Continue.
[DEBUG]: Checksums: []
[WARN]: No checksum is found! This is discouraged!
[DEBUG]: Source location: /var/cache/acbs/tarballs/glib-2.50.2.tar.xz, Shadow link: /var/cache/acbs/build/acbs.rri5k2a9/glib-2.50.2.tar.xz
[WARN]: ACBS cannot find libmagic bindings, will use bundled one instead.
[DEBUG]: Traceback:
  File "/bin/acbs-build", line 91, in <module>
    main()
  File "/bin/acbs-build", line 42, in main
    acbs_instance.build()
  File "/usr/lib/acbs/acbs/mainflow.py", line 103, in build
    self.build_single_pkg(matched_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 169, in build_single_pkg
    self.build_main(single_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 148, in build_main
    self.new_build_thread(try_build)
  File "/usr/lib/acbs/acbs/mainflow.py", line 193, in new_build_thread
    'Sub-build process building \033[36m{}\033[0m \033[93mfailed!\033[0m'.format(sub_pkg)) from ex

Traceback (most recent call last):
  File "/usr/lib/acbs/acbs/src_process.py", line 60, in file_type
    import magic
ModuleNotFoundError: No module named 'magic'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/acbs/acbs/mainflow.py", line 190, in new_build_thread
    return sub_thread.get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/acbs/acbs/mainflow.py", line 179, in slave_thread_build
    return new_build_instance.build()
  File "/usr/lib/acbs/acbs/mainflow.py", line 103, in build
    self.build_single_pkg(matched_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 169, in build_single_pkg
    self.build_main(single_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 154, in build_main
    tmp_dir_loc.append(SourceProcessor(self.pkg_data, self).process())
  File "/usr/lib/acbs/acbs/src_process.py", line 55, in process
    self.decomp_file()
  File "/usr/lib/acbs/acbs/src_process.py", line 95, in decomp_file
    file_type_name = self.file_type()
  File "/usr/lib/acbs/acbs/src_process.py", line 64, in file_type
    import acbs.magic as magic
  File "/usr/lib/acbs/acbs/magic.py", line 61, in <module>
    _open = _libraries['magic'].magic_open
  File "/usr/lib/python3.6/ctypes/__init__.py", line 357, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 362, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python3: undefined symbol: magic_open

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/bin/acbs-build", line 91, in <module>
    main()
  File "/bin/acbs-build", line 42, in main
    acbs_instance.build()
  File "/usr/lib/acbs/acbs/mainflow.py", line 103, in build
    self.build_single_pkg(matched_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 169, in build_single_pkg
    self.build_main(single_pkg)
  File "/usr/lib/acbs/acbs/mainflow.py", line 148, in build_main
    self.new_build_thread(try_build)
  File "/usr/lib/acbs/acbs/mainflow.py", line 193, in new_build_thread
    'Sub-build process building \033[36m{}\033[0m \033[93mfailed!\033[0m'.format(sub_pkg)) from ex
acbs.utils.ACBSGeneralError: Sub-build process building glib failed!

TODO

This is a refactor of its ancestor, it still contains a part of its ancestor's code.

Total code cleanup
Thorough and extensive testing
Dinner served, *******s!

flow: build fails with a package group in sequence

Command: acbs-build passlib groups/gstreamer-1-0 (still reproducible).

Log:

Preparing to unpack passlib_1.7.1-0_noarch.deb ...
Unpacking passlib (1.7.1) ...
Setting up passlib (1.7.1) ...
[INFO]: >>>>>>>>> Time for building passlib: 0:01:04.643005
[INFO]: Package build list found: gstreamer-1-0 (9)
[INFO]: Start building ../../../../var/cache/acbs/build/acbs.jadwlya9/passlib-1.7.1/passlib
[WARN]: Unknown folder: tests in tree
[WARN]: Unknown folder: handlers in tree
[WARN]: Unknown folder: _setup in tree
[WARN]: Unknown folder: crypto in tree
[WARN]: Unknown folder: __pycache__ in tree
[WARN]: Unknown folder: utils in tree
[WARN]: Unknown folder: ext in tree
[WARN]: Unknown folder: _data in tree

[CRIT]: Oops! ValueError: No vaild package found!

Cycle detection in deep dep checking, if there is one

AOSC-Archive/abbs#16, abbs@e0a4e6e.

Code review

Handle when default shell is not bash
- Provide a clean way to call bash
Remove unnecessary calls to external programs
- Checksum: hashlib
- Download: requests
os.path.join and os.path.split
Should configure more directories
Cleanup hacks

	logging.warning(
	'Failed to load libarchive library! Fall back to bsdtar!')

aosc-dev / acbs Goto Github PK

acbs's People

Contributors

Stargazers

Watchers

Forkers

acbs's Issues

Recommend Projects

Recommend Topics

Recommend Org