Giter VIP home page Giter VIP logo

synocommunity / spksrc Goto Github PK

View Code? Open in Web Editor NEW
3.0K 169.0 1.2K 67.07 MB

Cross compilation framework to create native packages for the Synology's NAS

Home Page: https://synocommunity.com

License: Other

Makefile 54.63% PHP 0.39% C++ 7.47% Perl 1.20% Shell 20.08% C 0.73% SuperCollider 0.17% Python 3.08% JavaScript 9.21% HTML 1.71% CSS 0.60% NewLisp 0.06% Ruby 0.12% Smarty 0.03% Go 0.41% Dockerfile 0.09% Roff 0.01% Vim Script 0.01% Scala 0.01%

spksrc's Introduction

Discord

SynoCommunity is now on Discord!

Discord

spksrc

spksrc is a cross compilation framework intended to compile and package software for Synology NAS devices. Packages are made available via the SynoCommunity repository.

DSM 7

DSM 7 was released on June 29 2021 as Version 7.0.41890.

  • The main issue we had with our reposity is fixed in spkrepo and online since February 2024
    • before the repository deliverd DSM 6 packages for Systems with DSM 7, when no DSM 7 package was available
    • this gave errors like "invalid file format" (or "package requires root privileges")
    • you still get this error when manually installing a DSM 6 package on DSM 7
  • You find the status of the former packages in the issue #4524 Meta: DSM7 package status
  • New packages support DSM 7 from initial package version (and some require at least DSM 7).
  • ATTENTION: As reported, package configuration settings may be lost following the upgrade to DSM 7 and the execution of a Package repair. Make sure to backup your settings and configuration for your SynoCommunity packages before installation of DSM 7 to facilitate restoration if needed.

Contributing

Before opening a new issue, check the FAQ and search open issues. If you can't find an answer, or if you want to open a package request, read CONTRIBUTING to make sure you include all the information needed for contributors to handle your request.

Setup Development Environment

Docker

The Docker development environment supports Linux and macOS systems, but not Windows due to limitations of the underlying file system.

  1. Fork and clone spksrc: git clone https://github.com/YOUR-USERNAME/spksrc
  2. Install Docker on your host OS (see Docker installation, or use a wget-based alternative for linux Install Docker with wget).
  3. Download the spksrc Docker container: docker pull ghcr.io/synocommunity/spksrc
  4. Run the container with the repository mounted into the /spksrc directory with the appropriate command for your host Operating System:
cd spksrc # Go to the cloned repository's root folder.

# If running on Linux:
docker run -it --platform=linux/amd64 -v $(pwd):/spksrc -w /spksrc ghcr.io/synocommunity/spksrc /bin/bash

# If running on macOS:
docker run -it --platform=linux/amd64 -v $(pwd):/spksrc -w /spksrc -e TAR_CMD="fakeroot tar" ghcr.io/synocommunity/spksrc /bin/bash
  1. From there, follow the instructions in the Developers HOW TO.

Virtual machine

A virtual machine based on an 64-bit version of Debian 11 stable OS is recommended. Non-x86 architectures are not supported.

Install the requirements (in sync with Dockerfile):

sudo dpkg --add-architecture i386 && sudo apt-get update
sudo apt update
sudo apt install autoconf-archive autogen automake autopoint bash bc bison \
                 build-essential check cmake curl cython3 debootstrap ed expect fakeroot flex \
                 g++-multilib gawk gettext git gperf imagemagick intltool jq libbz2-dev libc6-i386 \
                 libcppunit-dev libffi-dev libgc-dev libgmp3-dev libltdl-dev libmount-dev libncurses-dev \
                 libpcre3-dev libssl-dev libtool libunistring-dev lzip mercurial moreutils ninja-build \
                 patchelf php pkg-config python2 python3 python3-distutils rename ruby-mustache rsync scons subversion \
                 swig texinfo unzip xmlto zip zlib1g-dev
wget https://bootstrap.pypa.io/pip/2.7/get-pip.py -O - | sudo python2
sudo pip2 install wheel httpie
wget https://bootstrap.pypa.io/get-pip.py -O - | sudo python3
sudo pip3 install meson==1.0.0

From there, follow the instructions in the Developers HOW TO.

  • You may need to install some packages from testing like autoconf. Read about Apt-Pinning to know how to do that.
  • Some older toolchains may require 32-bit development versions of packages, e.g. zlib1g-dev:i386

LXC

A container based on 64-bit version of Debian 11 stable OS is recommended. Non-x86 architectures are not supported. The following assumes your LXD/LXC environment is already initiated (e.g. lxc init) and you have minimal LXD/LXC basic knowledge :

  1. Create a new container (will use x86_64/amd64 arch by default): lxc launch images:debian/11 spksrc
  2. Enable i386 arch: lxc exec spksrc -- /usr/bin/dpkg --add-architecture i386
  3. Update apt channels: lxc exec spksrc -- /usr/bin/apt update
  4. Install all required packages:
lxc exec spksrc -- /usr/bin/apt install autoconf-archive autogen automake autopoint bash bc bison \
                                build-essential check cmake curl cython3 debootstrap ed expect fakeroot flex \
                                g++-multilib gawk gettext git gperf imagemagick intltool jq libbz2-dev libc6-i386 \
                                libcppunit-dev libffi-dev libgc-dev libgmp3-dev libltdl-dev libmount-dev libncurses-dev \
                                libpcre3-dev libssl-dev libtool libunistring-dev lzip mercurial moreutils ninja-build \
                                patchelf php pkg-config python2 python3 python3-distutils rename rsync ruby-mustache scons subversion \
                                swig texinfo unzip xmlto zip zlib1g-dev
  1. Install python2 wheels:
lxc exec spksrc -- /bin/bash -c "wget https://bootstrap.pypa.io/pip/2.7/get-pip.py -O - | python2"
lxc exec spksrc -- /bin/bash -c "pip2 install virtualenv httpie"
  1. Install python3 pip:
lxc exec spksrc -- /bin/bash -c "wget https://bootstrap.pypa.io/get-pip.py -O - | python3"
  1. Install meson:
lxc exec spksrc -- /bin/bash -c "pip3 install meson==1.0.0"

LXC: spksrc user

  1. By default it is assumed that you will be running as spksrc user into the LXC container. Such user needs to be created into the default container image:
lxc exec spksrc -- /usr/sbin/adduser --uid 1001 spksrc
  1. Setup a default shell environment:
lxc exec spksrc --user 1001 -- cp /etc/skel/.profile /etc/skel/.bashrc ~spksrc/.

From there you can connect to your container as spksrc and follow the instructions in the Developers HOW TO.

lxc exec spksrc -- su --login spksrc
spksrc@spksrc:~$

(OPTIONAL) Install misc base tools:

lxc exec spksrc -- /usr/bin/apt install bash-completion man-db manpages-dev \
                                        mlocate ripgrep rsync tree time
lxc exec spksrc -- /usr/bin/updatedb

Install github client:

$ lxc exec spksrc -- su --login root
# curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
# echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
# sudo apt update
# sudo apt install gh
# exit

(OPTIONAL) LXC: Shared spksrc user

You can create a shared user between your Debian/Ubuntu host and the LXC Debian container which simplifies greatly file management between the two. The following assumes you already created a user spksrc with uid 1001 in your Debian/Ubuntu host environment and that you which to share its /home userspace.

  1. Create a mapping rule between the hosts and the LXC image:
lxc config set spksrc raw.idmap "both 1001 1001"
lxc restart spksrc
Remapping container filesystem
  1. Add /home/spksrc from the hsot to the LXC container:
lxc config device add spksrc home disk path=/home/spksrc source=/home/spksrc
Device home added to spksrc
  1. Connect as spksrc user:
lxc exec spksrc -- su --login spksrc
spksrc@spksrc:~$

LXC: Proxy (OPTIONAL)

The following assume you have a running proxy on your LAN setup at IP 192.168.1.1 listening on port 3128 that will allow caching files.

  1. Enforce using a proxy:
lxc config set spksrc environment.http_proxy http://192.168.1.1:3128
lxc config set spksrc environment.https_proxy http://192.168.1.1:3128
  1. Enforce using a proxy with wget in the spksrc container user account:
lxc exec spksrc --user $(id -u spksrc) -- bash -c "cat << EOF > ~spksrc/.wgetrc
use_proxy = on
http_proxy = http://192.168.1.1:3128/
https_proxy = http://192.168.1.1:3128/
ftp_proxy = http://192.168.1.1:3128/
EOF"

Usage

Once you have a development environment set up, you can start building packages, create new ones, or improve upon existing packages while making your changes available to other people. See the Developers HOW TO for information on how to use spksrc.

License

When not explicitly set, files are placed under a 3 clause BSD license

spksrc's People

Contributors

blackjid avatar bru7us avatar cytec avatar dependabot[bot] avatar diaoul avatar dr-bean avatar fgma avatar filin20 avatar gaetancambier avatar hgy59 avatar hmflash avatar joenyland avatar karaokestu avatar kmarty avatar lost-carrier avatar m4tt075 avatar manulin44 avatar maxrogers avatar moneytoo avatar mreid-tt avatar patsissons avatar piejanssens avatar piwi82 avatar publicarray avatar safihre avatar seblucas avatar smaarn avatar th0ma7 avatar ymartin59 avatar zebulon501 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spksrc's Issues

Add "enable https" feature in NZB Config

A helper to configure HTTPS for SABnzbd / SickBeard / CouchPotato / Headphones could be useful as default certificates are not readable with the dedicated user.
This would copy certs into the var directory with correct persmissions.

python26 and 27 do not compile on Ubuntu 11.11

make ARCH=88f6281 python26 returned this at the end :

 Failed to find the necessary bits to build these modules:
_bsddb             _hashlib           _tkinter        
bsddb185           bz2                dbm             
gdbm               linuxaudiodev      ossaudiodev     
sunaudiodev                                           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_curses_panel                                         

mkdir ./Lib/plat-linux3
cp ./Lib/plat-generic/regen ./Lib/plat-linux3/regen
export PATH; PATH="`pwd`:$PATH"; \
    export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; \
    export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; \
    export EXE; EXE=""; \
    cd ./Lib/plat-linux3;  ./regen
python$EXE ../../Tools/scripts/h2py.py -i '(u_long)' /usr/include/netinet/in.h
/home/antoine/Documents/spksrc/spk/python26/work-88f6281/Python-2.6.6/python: 1: Syntax error: word unexpected (expecting ")")

HDD's never hibernate

So I'm using SABnzbd, Couch Potato, Sick Beard, Headphones, but noticed my HDD's never hibernate...

They used to hibernate when I never had the above packages installed, but now they seems to be always on.
Im not sure if there is a setting in each application or what, so maybe make it search like each day, or something to let the HDD's sleep.

Thoughts?

Can't compile not pure python modules

For example setuptools requires a shared lib /home/spksrc/spksrc/spk/python27/work-88f6281/Python-2.7.2/build/lib.linux-i686-2.7/time.so which is made for the requested target arch (88f6281in my case) not host arch (x86 in my case)

Traceback (most recent call last):
  File "setup.py", line 12, in <module>
    from setuptools import setup, find_packages
  File "/home/spksrc/spksrc/spk/python27/work-88f6281/setuptools-0.6c11/setuptools/__init__.py", line 2, in <module>
    from setuptools.extension import Extension, Library
  File "/home/spksrc/spksrc/spk/python27/work-88f6281/setuptools-0.6c11/setuptools/extension.py", line 2, in <module>
    from dist import _get_unpatched
  File "/home/spksrc/spksrc/spk/python27/work-88f6281/setuptools-0.6c11/setuptools/dist.py", line 6, in <module>
    from setuptools.command.sdist import sdist
  File "/home/spksrc/spksrc/spk/python27/work-88f6281/setuptools-0.6c11/setuptools/command/sdist.py", line 5, in <module>
    import os, re, sys, pkg_resources
  File "/home/spksrc/spksrc/spk/python27/work-88f6281/setuptools-0.6c11/pkg_resources.py", line 16, in <module>
    import sys, os, zipimport, time, re, imp
ImportError: /home/spksrc/spksrc/spk/python27/work-88f6281/Python-2.7.2/build/lib.linux-i686-2.7/time.so: cannot open shared object file: No such file or directory
make[2]: *** [myCompile] Erreur 1
make[2]: quittant le répertoire « /home/spksrc/spksrc/cross/setuptools »
make[1]: *** [depend_target] Erreur 2
make[1]: quittant le répertoire « /home/spksrc/spksrc/spk/python27 »
make: *** [python27] Erreur 2

Tried to set LD_LIBRARY_PATH without luck.
I'll try to fix that copying native libs to work dir temporary while installing a module. So I fake python.

Use a REV number with the same amount of digits

Because it's the way Synology does it.
Because I think that Package Center won't be able to compare REV 1 with REV 10 of a SPK (lets say Python 2.7.2):

2.7.2-1 == 2.7.2-10
Whereas
2.7.2-001 < 2.7.2-010

Need to test this

julius rpath is not correct

@zebulon501: what's the relation between parameters given to ./configure and default rpath used for binaries at compile time? Sometimes it seems that -Wl,-rpath is ignored (julius is an example). Is there an easy way to change rpath afterward?

I've ran into this issue already with syno-packager and used http://linux.die.net/man/1/chrpath but I think when we discussed about this issue a few time ago you had another tool to manipulate rpath?

Cannot update SB

SB is showing an update is available.
When clicking on it I get this:
Update Failed
Update wasn't successful, not restarting. Check your log for more information.

And in the log it says this:
2011-12-09 11:10:14.747813 CP Server Thread-5 :: Unable to retrieve new version from http://github.com/midgetspy/Sick-Beard/tarball/master, can't update.

Is that just a SB (Github) issue?

dsm-control.sh

Hi,

i have forked spksrc in order to do cp v2 package. I got almost everything working, except for my start script which is not created upon package installation?
any idea?

Thanks

Backup issue with Python package (is this spksrc related?)

Hi
since the update to DSM 3.2-1944, I am not able to perform network backup to rsync server. It fails when backing up apps:

Nov 4 21:25:43 PkgMan.cgi: pkgtool.cpp:2198 No script for Python26
Nov 4 21:25:50 synonetbkp: sysbkp_user_export.c:94 fail to write user data
Nov 4 21:25:50 synonetbkp: SYNOSysbkpExport:Fail to export user_desc
Nov 4 21:25:50 synonetbkp: backup_app_apply.c:87 Failed to export application [1], bkptype=[0], szAppPath=[/volume1/@tmp/@app],
synoerr=[0x5500]
Nov 4 21:25:50 synonetbkp: synonetbkp.c:1991 SYNOBackupAppApply() failed!! ulExtension=[1], mode=[MODE_BACKUP], bkptype=
[BACKUP_TYPE_NETBKP], synoerr=[0x5500]

I am not sure whether this is Python package problem (from http://synopkg.superzebulon.org/spkrepo/packages) or something else... More users report this problem, see http://synoblog.superzebulon.org/2011/10/some-news/#comment-8874)

Cheers!

Port numbers of SPKs

In my branch I leave defaults port as much as possible if those ports are listed here : http://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers and do not conflict with Synology port numbers http://www.synology.com/support/faq_show.php?q_id=299&lang=enu

For example, Sick Beard default port AFAIR is 8081which is a registered.

Thus I thought about a port range for web UIs with DSM :
82xy
x being unique per SPK
y being 0 for HTTP protocol, 1 for HTTPS

What do you think @zebulon501 ? Should we leave default port numbers all the time as long as it does not conflict with Synology ones ? Or should we care about registered port numbers ?
Are you ok with 82xy range if no default port is specified (ZNC for example) ?

No users created

Another thing I just noticed.
Once I installed all these packages on a few Syno, it has not created any users.

I know your packages used to create a user for each package.
Right now its only showing my default user which is 'admin'.

If its not supposed to create users any more that's fine, just thought I would check in with you.

SABnzbd is not showing as Running...

I have a fresh install of your packages.
They are ALL working fine, except one small thing.

SABnzbd is not showing as running in the Package Centre.
Has the option to Run, but it is already up and running.

I have restarted the DSM, and it still doesnt show as running.

Ps im using a RackStation RS810+ if thats any help.

Create a rule to fix paths for python scripts

For example, when installing python module pip, the shebang is #!$(HOSTPYTHON).
We need to replace all created scripts with #!/usr/local/python2x/bin/python

This could be done with sed each time a module is installed: check for PLIST and do the appropriate action for the bin

I'll look into this solution.

Change log feature

spksrc should handle changelog files and auto generate them.

  • A target "changelog" could be created in the spk.mk that would produce the changelog in the console.
  • A CHANGELOG variable would point to the changelog file to send when publishing

openssl: libs don't contain an RPATH value

Using readelf on the shared library provided by OpenSSL, no RPATH value is shown. This will lead to some wrong dependencies (using libs in /lib and /usr/lib in place of the intended ones).

Create NZB Configurator SPK

Easier to maintain & manage than multiple configurators in each SPK.
The goal is to:

  • Configure SickBeard, CouchPotato, Headphones for either SABnzbd or NZBGet (SPK coming soon)
  • Have a nice UI like VPN Server with different panels for each installed applications to configure
  • User can choose adminonly or not

Packages work on 4.0 beta?

Do these packages continue to work on 4.0 beta?
If I upgraded to 4.0 DSM would I need to reinstall or will they continue to work with all my settings during the upgrade?

Also are you still maintaining these packages as it seems very quiet lately?

Allow parallel make execution

This is possible with the make -j option.
I tried but it fails at some points of the code because not all targets can be parallelized.

I see two solutions to that:

  • Build our own parallel stuff (by adding PARALLEL=1 to local.mk for example) and make the code parallel ourselves in the code
  • Use the builtin -j option that comes with make but we have to specify .NOTPARALLEL where needed

I recommend the second option, that's the "standard" way.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Java package

I have a few Ruby apps that I am building which are targeted at the Synology NASes. Instead of having to build Ruby, gems, etc, I'm thinking of building them with JRuby and then compiling the app to a .jar file for distribution.

I'm putting this request in here for a separate Java spk. I may in fact give it a shot myself, but it isn't totally clear to me (yet) how to use spksrc or what OSes I can compile things on.

I can see somebody's (successful?) attempt at getting Java on the Synology OS here http://pcloadletter.co.uk/2011/08/23/java-package-for-synology/. It makes sense to me to have this become the canonical place for finding 3rd party SPKs, though, since you've done such an excellent job up until now.

What to do with packages that includes a python module?

This is the case with speechd: 40b70ea

I think a SPK with speechd should not interfere with python2x SPK if installed and should be shipped with it's own Python installation if the python module has to be used at some point.
However, it is still difficult to handle.

What if we want to include them in Python SPK? What's right way to do it? I did a workaround here: https://github.com/SynoCommunity/spksrc/blob/speech/cross/sphinxbase/Makefile

Improve SABnzbd

  • Check daemon status with pid or with ps | grep
  • Stop daemon with pid file

This will avoid HTTP requests and avoid useless CPU load
Some stuff can be taken from here

Migration Framework

The idea would be to create an standardized migration framework.

Say you have rev 2 on your computer and rev 4 is out. Package Center will not do 2 -> 3 -> 4 but directly 2 -> 4. That might be a problem if rev 3 introduces some fundamental changes to the structure.

In installer.sh just add some functions :

  • migrate_1_to_2
  • migrate_2_to_3
  • migrate_3_to_4

Add a little script in preupdate that will loop from the rev extracted from SYNOPKG_OLD_PKGVER and execute the appropriate functions in the right order.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

UI Develop SPK

Add a UI Develop SPK so its easy for developers to edit the correct files through a share and directly view the result in DSM.

Fix building of standard Python modules

Already done ncurses and bz2 but a few remain (Python 2.7):

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             _ssl               _tkinter        
bsddb185           dbm                gdbm            
sunaudiodev                                           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

At least _ssl requires a fix IMO, everything else isn't that important.

Use bindir, libdir in configure

I think those options are meant to define the place to install stuff during make install. Currently they aren't used.
OpenEmbedded uses this.

Also, if not used, make install will fail for some libs like libsndfile. A (workaround is used for libsndfile)[https://github.com/SynoCommunity/spksrc/commit/a9e874d82592ab8cd2c9be1952c494d2499d18d1] but is still buggy as install target does something like test -z /usr/local/xxx || mkdir -p /home/spksrc/spksrc/cross/libsndfile/.../use/local/xxx

@zebulon501: Any idea/suggestion about this? How to do this the right way?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.