Comments (44)
@raveit65 @sc0w The server was upgraded to Debian 10 yesterday 😃
from mate-dev-scripts.
You can use nginx or caddy or other software, that support reverse proxy, to use https like https://pub.mate-desktop.org/release as the notify server url.
from mate-dev-scripts.
This is the step by step to use the release-server
:
- download the last
release-server
binary from here:
https://github.com/yetist/release-server/releases
then save therelease-server
in $PATH directory and give it execute permission. - download the example config file from here:
https://github.com/yetist/release-server/blob/master/release-server.toml.example
and rename it torelease-server.toml
, put it in/etc/xdg/release-server/<VERSION>/release-server.toml
- edit the
release-server.toml
file.
In the[path]
section, to setup where the tarballs saved, these path maybe used in static http server, such as apache, nginx or caddy.
In the[security]
section, there are there item need to set:
a)api_secret
: please use some random characters. Then, goto the travis website, select one repo which want to use auto release, create new hide environment variable namedAPI_SECRET
and set it to use the the same value.
b)allow_repos
can simple setup to*
for allow any repo, or write every repos likemate-desktop/caja
,mate-desktop/mozo
etc.
c)allow_ips
can setup to the travis-ci ip address, they can found here: https://docs.travis-ci.com/user/ip-addresses/ - run
release-server
in the server. - goto the repo which to use github release page, append the
notify_servers
into.travis.yml
.
The example is here: https://github.com/yetist/mate-common/blob/master/.travis.yml
The url is provide by therelease-server
, according the therelease-server.toml
.
Then, if you create tag for some repo, and push it to the github, the github release page will created, and the tarballs will saved in the custom release server.
You can try it and hope to go well.
from mate-dev-scripts.
The pre-release folder wasn't a public folder. We use it in the past to share tarballs for testing between team members.
And is there a way to upload tarballs by hand? Same what we did here https://release.mate-desktop.org
People with access to this website cold upload a tarball there, which was moved to public download folder internally.
from mate-dev-scripts.
This is two questions:
- Publish drafts and pre-release versions:
Github supports the release of drafts, pre-releases, and release versions. To automate them, you need to enable the settings in.travis.yml
:
releases:
draft: false
prerelease: false
...
- Custom server listing directory:
Therelease-server
only download tarballs from github release page and save them into some directories on the server.
Whether to display these directories publicly, or to require authentication to access these directories, can be changed by modifying the http server configuration.
from mate-dev-scripts.
The whole process is like this:
local -> travis -> github release -> http server.
local: git push TAG =>
travis: make dist -> publish to github release -> notify http server =>
http server: download tarballs from github -> save them into static http directory =>
person: visit url, to see the tarballs in http server.
from mate-dev-scripts.
Ok, so the folder stucture is what we used at github release page and doesn't have anything to do with our old pre-relase and release folder to upload tarballs to our server.
then save the release-server in $PATH directory and give it execute permission.
This is /usr/bin ? Or can i put the binary in a home folder of a fresh account?
I think a systemd unit file would be nice and comfortabel for starting the binary.
Hopefully i will find some time over the weekend to configure the server.
from mate-dev-scripts.
Yes, you can set the directory as needed.
You can also use the old method instead of the draft and pre-release provided by github.
The release-server
can be placed in any suitable directory as long as systemd can start it.
Here is demo for that:
[Unit]
Description=Release server
After=syslog.target
After=network.target
[Service]
LimitNOFILE=65535
RestartSec=2s
Type=simple
User=www-data
Group=www-data
ExecStart=/usr/bin/release-server
Restart=always
[Install]
WantedBy=multi-user.target
from mate-dev-scripts.
@yetist
Which characters can I use for the api_secret token?
Letters, numbers, upper and lower case, special characters ?
from mate-dev-scripts.
Letters, numbers, upper and lower case is ok, except space and Tab.
from mate-dev-scripts.
One question regarding the new release system. The current system updates an RSS feed when a new release is published:
New feed entries are automatically posted to social media. Where will the RSS feed reside for the new release system?
from mate-dev-scripts.
According to your configuration files, https://pub.mate-desktop.org/releases
is used to receive
the notification, but the real service address is http://<IP>:9090/release
, Do you setup the reverse proxy so that they access the same service? If not, the notify_server
should be set to http://<IP>:9090/release
.
I used caddy, the configuration file is as follows:
# vim: set filetype=caddy ts=4 sts=4 :
mate.zhcn.cc {
root /srv/http/mate-desktop/
log /var/log/caddy/mate-desktop.log
gzip
index index.html
browse
}
post.zhcn.cc {
log /var/log/caddy/mate-desktop.log
proxy / http://localhost:9090
limits 20mb
}
The release-server's config file:
host = "localhost"
port = 9090
So I can use the https://post.zhcn.cc/release
as the notify_servers
in .travis.yml:
notify_servers:
- https://post.zhcn.cc/release
from mate-dev-scripts.
Please check the server with:
$ curl -v -d 'a=b' https://pub.mate-desktop.org:9090/release
If a valid service is tested, it responds {"status":"invalid request."}
:
$ curl -v -d 'a=b' http://localhost:9090/release
* Trying ::1:9090...
* TCP_NODELAY set
* Connected to localhost (::1) port 9090 (#0)
> POST /release HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.66.0
> Accept: */*
> Content-Length: 3
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 3 out of 3 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 Bad Request
< Content-Type: application/json; charset=utf-8
< Date: Sat, 05 Oct 2019 01:20:58 GMT
< Content-Length: 29
<
* Connection #0 to host localhost left intact
{"status":"invalid request."}
from mate-dev-scripts.
It seems the server is running, but what is the result when you running curl at your working pc? If there is no result, maybe you should check the iptables at server.
from mate-dev-scripts.
So I think the firewall is open at server, and it disable the port, you should check that.
from mate-dev-scripts.
I've open port 9090 on the firewall. The other thing to consider is that the server is behind a CloudFlare CDN, mostly to block malicious traffic.
https traffic to mate-desktop.org
will be terminated by the CloudFlare CDN. So we need to connect to the release server on a FQDN that is not terminated by the CDN, yerba.mate-desktop.org
should work.
from mate-dev-scripts.
If possible, I recommend turning off port 9090, which can be done using a reverse proxy. And I recommend using https without a port to access it.
from mate-dev-scripts.
@yetist Understood 👌 I'll use nginx.
from mate-dev-scripts.
@raveit65 @yetist The release-server is now behind an nginx reverse proxy and available via https://release.mate-desktop.org
from mate-dev-scripts.
@raveit65 @yetist The release-server is now behind an nginx reverse proxy and available via https://release.mate-desktop.org
OK, https://release.mate-desktop.org/release works.
from mate-dev-scripts.
I have closed port 9090 on the server.
I have also shutdown the old release services.
If you like, you can use some authentication and not allow public access to https://pub.mate-desktop.org/draft/ or https://pub.mate-desktop.org/pre-release/.
The config in .travis.yml can control that:
releases:
draft: false
prerelease: false
EDIT:
if draft: true
, the released tarball will saved into https://pub.mate-desktop.org/draft/, if prerelease: true
, the tarball saved into https://pub.mate-desktop.org/pre-release/
from mate-dev-scripts.
@yetist
sucessfull test with
notify_servers:
- https://release.mate-desktop.org/release
and
allow_repos = [
"*"
]
Edit: with mate-common-1.23.150
from mate-dev-scripts.
Can you test it again to use, please?
allow_repos = [
"raveit65/mate-common"
]
For more infomation, you can stop systemd release-server, and running release-server
by hand on server, let's see the output messages.
from mate-dev-scripts.
I think I found the reason about allow_repos, I made a mistake here: https://github.com/yetist/release-server/blob/master/util.go#L28
https: //github.com
, after https:
, I type a space, this space should be removed.
from mate-dev-scripts.
For release-server 0.1.4, it use the config here: /etc/xdg/release-server/0.1.4/release-server.toml
, dont forget to rename the 0.1.3
directory.
from mate-dev-scripts.
The release-server 0.1.4 binary is ready: https://github.com/yetist/release-server/releases/tag/v0.1.4
from mate-dev-scripts.
@yetist
Thanks a lot, you fixed the issue with 0.1.4
mate-common 1.23.152 was done with
allow_repos = [
"raveit65/mate-common"
]
https://pub.mate-desktop.org/releases/1.23/
from mate-dev-scripts.
root@yerba:/home/web-release/release-server-travis-github/0.1.4# ls -l /home/web-release/pub.mate-desktop.org/sources/mate-common/1.23/
total 192
lrwxrwxrwx 1 web-release web-release 78 Sep 26 14:40 mate-common-1.23.0.tar.xz -> /home/web-release/pub.mate-desktop.org/releases/1.23/mate-common-1.23.0.tar.xz
-rw-r--r-- 1 web-release web-release 41 Sep 26 14:40 mate-common-1.23.0.tar.xz.sha1sum
-rw-r--r-- 1 web-release web-release 42 Oct 5 14:38 mate-common-1.23.150.news
-rw-r--r-- 1 web-release web-release 69308 Oct 5 14:38 mate-common-1.23.150.tar.xz
-rw-r--r-- 1 web-release web-release 94 Oct 5 14:38 mate-common-1.23.150.tar.xz.sha256sum
-rw-r--r-- 1 web-release web-release 42 Oct 5 15:45 mate-common-1.23.152.news
-rw-r--r-- 1 web-release web-release 69184 Oct 5 15:45 mate-common-1.23.152.tar.xz
-rw-r--r-- 1 web-release web-release 94 Oct 5 15:45 mate-common-1.23.152.tar.xz.sha256sum
lrwxrwxrwx 1 web-release web-release 78 Sep 26 21:10 mate-common-1.23.1.tar.xz -> /home/web-release/pub.mate-desktop.org/releases/1.23/mate-common-1.23.1.tar.xz
-rw-r--r-- 1 web-release web-release 41 Sep 26 21:10 mate-common-1.23.1.tar.xz.sha1sum
lrwxrwxrwx 1 web-release web-release 78 Sep 28 09:05 mate-common-1.23.2.tar.xz -> /home/web-release/pub.mate-desktop.org/releases/1.23/mate-common-1.23.2.tar.xz
-rw-r--r-- 1 web-release web-release 41 Sep 28 09:05 mate-common-1.23.2.tar.xz.sha1sum
lrwxrwxrwx 1 web-release web-release 78 Sep 30 18:05 mate-common-1.23.3.tar.xz -> /home/web-release/pub.mate-desktop.org/releases/1.23/mate-common-1.23.3.tar.xz
-rw-r--r-- 1 web-release web-release 41 Sep 30 18:05 mate-common-1.23.3.tar.xz.sha1sum
root@yerba:/home/web-release/release-server-travis-github/0.1.4# ls -l /home/web-release/pub.mate-desktop.org/releases/1.23/
<cut>
-rw-rw-r-- 1 release sftponly 69528 Sep 26 14:36 mate-common-1.23.0.tar.xz
-rw-r--r-- 1 web-release web-release 42 Oct 5 14:38 mate-common-1.23.150.news
-rw-r--r-- 1 web-release web-release 69308 Oct 5 14:38 mate-common-1.23.150.tar.xz
-rw-r--r-- 1 web-release web-release 94 Oct 5 14:38 mate-common-1.23.150.tar.xz.sha256sum
-rw-r--r-- 1 web-release web-release 42 Oct 5 15:45 mate-common-1.23.152.news
-rw-r--r-- 1 web-release web-release 69184 Oct 5 15:45 mate-common-1.23.152.tar.xz
-rw-r--r-- 1 web-release web-release 94 Oct 5 15:45 mate-common-1.23.152.tar.xz.sha256sum
-rw-rw-r-- 1 release sftponly 69632 Sep 26 21:07 mate-common-1.23.1.tar.xz
-rw-rw-r-- 1 release sftponly 69584 Sep 28 09:01 mate-common-1.23.2.tar.xz
-rw-rw-r-- 1 release sftponly 69380 Sep 30 17:59 mate-common-1.23.3.tar.xz
<cut>
@yetist
Do you think it is possible to use symlinks for tarballs in pub.mate-desktop.org/sources/ ?
Same like we did for older releases.
from mate-dev-scripts.
@yetist
Do you think it is possible to use symlinks for tarballs in pub.mate-desktop.org/sources/ ?
Same like we did for older releases.
Done, yetist/release-server@c70ef21
from mate-dev-scripts.
Cool :)
I forgot to ask......
Does the release server handle different directory on server for mate-themes?
They are located under https://pub.mate-desktop.org/releases/themes/ .
from mate-dev-scripts.
Ok, i deleted some posts (mostly mine) to make this topic readable.
So this can be used as a HOWTO.
from mate-dev-scripts.
New version release-server-0.1.6 is ready: https://github.com/yetist/release-server/releases/tag/v0.1.6
It can handle themes and update rss.xml file, rss have some config items, please see the example: https://github.com/yetist/release-server/blob/master/release-server.toml.example.
EDIT:
For mate-themes, I suggest to use
url_prefix = "https://pub.mate-desktop.org/sources/{{.Name}}/{{.ApiVersion}}"
in config file, they has the same rules.
from mate-dev-scripts.
@yetist
You last edit of you post confused me....
I don't understand what i should write for mate-themes in release-server.toml ?
Isn't this line for rss-feed?
url_prefix = "https://pub.mate-desktop.org/sources/{{.Name}}/{{.ApiVersion}}"
from mate-dev-scripts.
0.1.6 version has two lines about 'url_prefix' in example file, after release 0.1.6, I removed the other line.
from mate-dev-scripts.
Got it, {{.mate-themes}}/{{.ApiVersion}}
are variables, so no edit is needed.
from mate-dev-scripts.
Please keep this line 'url_prefix = "https://pub.mate-desktop.org/sources/{{.Name}}/{{.ApiVersion}}"' in config file, the variables will use correct name by release-server.
Under the sources directory, they have the same rules, so we don't need to handle the mate-themes here.
from mate-dev-scripts.
I got it running. Stupid typo in release-server.toml.
allow_ips = [
"34.66.25.221","34.66.50.208","34.66.178.120","34.66.200.49","34.68.144.114",
"35.184.96.71","35.184.226.236","35.188.1.99","35.188.73.34","35.192.85.2",
"35.192.136.167","35.192.187.174","35.193.7.13","35.193.14.140","35.202.145.110",
"35.202.245.105","35.224.112.202","104.154.113.151","104.154.120.187","104.198.131.58""
]
Double double-quote at end of line :/
from mate-dev-scripts.
Cool, rss.xml works, screenshot from liferea:
And I think the old scripts should stop to update the rss.xml.
from mate-dev-scripts.
Yes, should be possible to disable old rss script.
Engrampa-1.23.2 was uploaded with caja via old sftp method and without the running new release-server.
That means your rss magic works when we upload tarballs by hand.
from mate-dev-scripts.
@flexiondotorg
Can you please remove rss parts from your release-runner.py script?
I don't want to break it ;)
from mate-dev-scripts.
I just released mate-power-manager 1.23.1 and i don't see a double entry in rss feed.
So, it might be Ok not to remove rss code from release-runner.py script?
I think we need it when uploading tarballs by hand.
Or does the release-server monitor /home/web-release/release.mate-desktop.org/uploads/
directory?
@yetist ^^^
from mate-dev-scripts.
I'd like to finish this topic and work on it. So again my questions.
@yetist
When i upload tarballs by hand via sftp to /home/web-release/release.mate-desktop.org/uploads/
,
does your release-server monitor this directory and create a new rss entry?
If true we can remove rss code from old release-runner.py script.
from mate-dev-scripts.
Release-server does not monitor the upload directory, so if you upload file by hand, it will not generate rss and checksum files. The rss content and checksum files are from the github release.
from mate-dev-scripts.
Ok, thanks for clarification.
Than it is better to leave the old release-runner.py script like it is, to monitor the upload directory for manual uploads.
Topic is resolved.
from mate-dev-scripts.
Related Issues (6)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mate-dev-scripts.