caddyserver / dist Goto Github PK

Ubuntu/Debian is the default OS in many VPS providers. It is nice to have a deb package for easy installation and keeping up to date. (Disclaimer: I am not familiar with debian packaging).

Provide shell completion scripts so users get their commands completed on click of tap. I found this tool to generate the scripts for Bash, Fish, and zsh from a single source of truth. I don't know if it meets all needs.

Hi,

I tried to install Caddy via apt on a Raspberry Pi 1+ according to the official docs. The install itself works, but the binary won't run because it doesn't match the architecture:

$ caddy --help
Illegal instruction

Same problem when downloading and installing by hand the armhf package. Which is really weird, because armhf is the correct architecture according the dpkg (dpkg --print-architecture => armhf). And, the filename contains armv7 which I also weird because I was certain that the Raspberry Pi 1 B+ was armv6.

So I went to GitHub release, downloaded and installed the armv6 release and it worked !

$ caddy | head
Caddy is an extensible server platform.

usage:
  caddy <command> [<args...>]

commands:
  adapt           Adapts a configuration to Caddy's native JSON
  build-info      Prints information about this build
  environ         Prints the environment
  file-server     Spins up a production-ready file server

So I think the armhf should actually point to the armv6 package?

My experience is primarily with nginx but I'm trying out Caddy for a new project. For some reason, I am unable to SFTP into my server.

For reference, I'm using DigitalOcean and I initiated the server with my SSH credentials.

Current official DEB repo does not provide GPG signed releases file, making it impossible to verify the authenticity of the repository. It seems that gemfury has no support for GPG signed releases file at all, so perhaps it would make sense to consider another provider - as this is a pretty essential thing to have?

There are certainly plenty of options to choose from. I have previously used Cloudsmith and it seems to be working just fine - @lskillen can perhaps pitch in here, I am not sure if a project of this size is something they can take on policy-wise.

It would be very helpful for Debian packaging if caddyserver/dist was tagged in sync with caddyserver/caddy.

Debian packages contain a watch-file that defines patterns, urls and transfer methods (ftp, https, git...) for fetching a specified or the latest version of upstream source code from one or more download locations.

Right now, the watch file would define to download the release that matches $version of the Debian package on https://github.com/caddyserver/caddy/tags and to fetch HEAD from https://github.com/caddyserver/dist.git.
For untagged repos, $version is derived from the commit. divergent versions in one package are then being concatenated with +~ in between. That means for Caddy that its Debian package would have version 2.4.3+~0.0~git20210810.d90cabd-1. Might make sense for some packages, but for this case it would only cause a lot of confusion for Debian users (and annoyance for maintainers)

Today it occured again, when i updated my ubuntu distro with
sudo apt update && sudo apt upgrade
using the official repos, there was also an caddy webserver update included from version caddy:amd64 (2.4.3 => 2.4.5)

The root dir in the Caddyfile is defined as
/usr/share/caddy
which is a symbolic link to the actual web project, including an index.html document.

After the update the content of /usr/share/caddy was emptied and replaced with the default caddy hello page content again. Had this problem a few months ago when updating as well and figured out that it has been linked to an caddy update.

I do not expect that an update deletes / replaces the content of the default root with the default hello caddy files.

The existing files should remain unmodified. (As they do on apache webserver for example)

Guess the update behaviour in the Debian/Ubuntu/Raspbian-Package is wrong.

To reproduce

• install an older version of caddy on ubuntu
• put your own index.html into /usr/share/caddy
• ensure caddy is configured to serve this document:

# Caddyfile
root * /usr/share/caddy

• ensure caddy is running and serving your custom index.html
• update caddy by running sudo apt update && sudo apt upgrade
• refresh your browser, the default page is displayed
• look into /usr/share/caddy - the default files have replaced the custom ones

1. What version of Caddy are you using (caddy -version)?

0.10.9

2. What are you trying to do?

man caddy

Having a man page available would be useful for quick reference on servers and would help comply with distro packaging guidelines.

I used repository from documentation for installation of Caddy.

echo "deb [trusted=yes] https://apt.fury.io/caddy/ /" \
    | sudo tee -a /etc/apt/sources.list.d/caddy-fury.list
sudo apt update
sudo apt install caddy

from this page:
https://caddyserver.com/docs/download

but now apt update ignore everything about that repo.
Also, if I try to open https://apt.fury.io looks down?

Some other repo or info?
Thank you.

Hi,

all webservers on debian systems (at least from official repositories) are running as user 'www-data'. In the Caddy Debian package it's 'caddy', per systemd service file.

IMO it would be better to adhere to Debian standards, if only to make it easier to switch from/to another webserver. F. ex. as a former nginx user, all my web pages are owned by www-data. Of course it's no much work to chown to user caddy, but inconvenient anyway.

Joachim

I am currently working on packaging Caddy for Debian.
IMO a lot of this repo should be packaged together with Caddy, but in order to include them in Debian they need a (free) license.
Also, I would be glad not to have to write them myself 😉

This is running in an LXC container, only purpose is Caddy, so running as root user.

root@caddy:~# systemctl status caddy
* caddy.service - Caddy
   Loaded: loaded (/etc/systemd/system/caddy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2020-06-06 14:44:18 CDT; 52s ago
     Docs: https://caddyserver.com/docs/
  Process: 119 ExecStart=/root/caddy run --config /root/Caddyfile2 --adapter caddyfile (code=exited, status=2)
 Main PID: 119 (code=exited, status=2)

Jun 06 14:44:18 caddy caddy[119]:         runtime/proc.go:1041
Jun 06 14:44:18 caddy caddy[119]: goroutine 1 [running]:
Jun 06 14:44:18 caddy caddy[119]: runtime.systemstack_switch()
Jun 06 14:44:18 caddy caddy[119]:         runtime/asm_amd64.s:330 fp=0xc00005e788 sp=0xc00005e780 pc=0x465320
Jun 06 14:44:18 caddy caddy[119]: runtime.main()
Jun 06 14:44:18 caddy caddy[119]:         runtime/proc.go:133 +0x70 fp=0xc00005e7e0 sp=0xc00005e788 pc=0x436c70
Jun 06 14:44:18 caddy caddy[119]: runtime.goexit()
Jun 06 14:44:18 caddy caddy[119]:         runtime/asm_amd64.s:1373 +0x1 fp=0xc00005e7e8 sp=0xc00005e7e0 pc=0x4672b1
Jun 06 14:44:18 caddy systemd[1]: caddy.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 06 14:44:18 caddy systemd[1]: caddy.service: Failed with result 'exit-code'.

This is in /etc/systemd/system/caddy.service:

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
ExecStart=/root/caddy run --config /root/Caddyfile2 --adapter caddyfile
ExecReload=/root/caddy reload --config /root/Caddyfile2 --adapter caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Running the command listed at ExecStart manually after boot works just fine.

Caddyfile:

(SecurityHeaders) {
        header {
                Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
                X-Xss-Protection "1; mode=block"
                X-Content-Type-Options "nosniff"
                X-Frame-Options "SAMEORIGIN"
                Content-Security-Policy "upgrade-insecure-requests"
                Referrer-Policy "strict-origin-when-cross-origin"
                Cache-Control "public, max-age=15, must-revalidate"
                Feature-Policy "accelerometer 'none'; ambient-light-sensor 'none'; autoplay 'self'; camera 'none'; encrypted-media 'none'; fullscreen 'self'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; midi 'none'; payment 'none'; picture-in-picture *; speaker 'none'; sync-xhr 'none'; usb 'none'; vr 'none'"
        }
}

sub.some.domain {
        import SecurityHeaders
        file_server
        reverse_proxy 192.168.1.120:6000
        tls {
                dns cloudflare token
        }
}

hi.some.domain {
        import SecurityHeaders
        @excludeDirs {
                not path /local/folder* 
        }
        file_server
        reverse_proxy @excludeDirs 192.168.1.139:8002
        tls {
                dns cloudflare token
        }
}

Currently, CentOS 9 Stream is not supported. Output of dnf copr enable @caddy/caddy:

Error: This repository does not have any builds yet so you cannot enable it now.

Would it be possible to have in this repo the script that creates the .deb package?

I want to provide a custom Caddy build with plugins and update my installation on my server with the package instead of having to do it manually.

Err:4 https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version Release
  404  Not Found [IP: 108.139.1.12 443]
...
E: The repository 'https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version Release' no longer has a Release file.

https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt from the install instructions does not seem to exist either, it returns 404.

On CentOS 7, I was unable to update from 2.7.4 to 2.7.5.

/var/tmp/rpm-tmp.joZmkn: line 1: fg: no job control
error: %pre(caddy-2.7.5-1.el7.x86_64) scriptlet failed, exit status 1
Error in PREIN scriptlet in rpm package caddy-2.7.5-1.el7.x86_64

I believe it is due to this commit: cc5eb55

Specifically, this line:

%sysusers_create_package is not defined in my /usr/lib/rpm/macros.d/macros.systemd file, and systemd-sysusers (which the macro would call) is not present.

I think the answer is either to add a fallback for CentOS 7, or for CentOS 7 users to get systemd-sysusers and %sysusers_create_package (I could not figure out how to get systemd-sysusers on my system), or to remove the package for CentOS 7.

--pidfile option was added through caddyserver/caddy#3235 but the option can only be specified through command line parameter which is not easy to do without modifying the systemd unit file, which is not practical when it gets overridden on package updates.

Can you add --pidfile /var/run/caddy.pid in the systemd unit file in the official package?

I'm opening this up as followup from mentioning this in Slack.

We've had some users in the forums recently that got snagged on SELinux rules preventing the caddy user from reading/writing files in /var/www/html and /var/log/caddy.

• https://caddy.community/t/centos-7-installation/18329/14
• https://caddy.community/t/caddy-erros-page-can-t-be-found-and-access-to-sub-domainname-com-was-denied/18408/8
• https://caddy.community/t/caddy-unable-to-access-certificates-after-upgrade-from-2-6-2-to-2-6-4/18989

Is there anything that can be done in the .rpm to smooth this over?

I was wondering if it was possible to add a "provides: nginx" or something like this to the caddy ubuntu packages?

Some packages require something like this:

$ apt-cache depends zammad
zammad
  ...
  Depends: curl
    curl:i386
 |Depends: elasticsearch
  Depends: <elasticsearch-oss>
 |Depends: nginx
    nginx-core
    nginx-extras
    nginx-full
    nginx-light
  Depends: apache2
 |Depends: postgresql
  ...

obviously the better solution would be to ask zammad to add |Depends: caddy to that block, but a) they have like >400 open issues on github, so I think that'd take longer and b) there might be other packages out there that would need that update too.

cheers 👋

caddy "add-package" will not persist upgrades

my idea would be to add a file that lists additional packages, which is read by the install-scripts or the init-scripts.

Hi!

so the issue is with the Debian ppa on gemfury.

It provides no InRelease or Release file. This file must contain a origin field (and maybe some others) so that unattended-upgrades can reference this as a "source" to which the auto updates are applied.

In Detail:

The Debian Repository Format specifies that there should be a InRelease file and in this file there are among others the optional fields:

• Origin
• Label
• Suite
• Codename

Unattended-upgrades is configured in the /etc/apt/apt.conf.d/50unattended-upgrades file. This file has two relevant sections like this:

Unattended-Upgrade::Allowed-Origins {
        "${distro_id}:${distro_codename}";
        "${distro_id}:${distro_codename}-security";
};

Unattended-Upgrade::Origins-Pattern {
        "origin=Debian,codename=${distro_codename},label=Debian";
        "origin=Debian,codename=${distro_codename},label=Debian-Security";
};

In the first one its of the form "<origin>:<archive>" and in the second one origin, codename, label and archive can be used. I am not exactly sure but I think archive is the same as Suite in the InRelease file.

The info about unattended-upgrades is from this blog post since the original wiki entry is not so detailed.

So initially this is just the issue with the missing InRelease/Release file but I can see how this turns out to be a bigger thing that might require a move away from gemfury. I don't know how you handle this stuff, I am not a Debian packaging pro, I just stumbled over this stuff while trying to setup and understand unattended-upgrades.

I am curious hearing what you think about that.

Best
Daniel

I just installed 2.7.6 from the cloudsmith repositories using the documentation instructions; my OS is Linux Mint 21.3.

Caddy fails with the following error: Error: reading config file: open onfig: no such file or directory

After digging around I understood that the root cause of the problem is in how the systemd unit file was written, it has: ExecStart=/usr/local/bin/caddy run -config /etc/caddy/Caddyfile (instead of --config, note the 2 dashes).

Having examined this repository, I see that the file says --config (i.e. it is correct), so I am not sure why I ended up with such a file. Maybe I looked in the wrong repository; in any case - I hope that this might ring some bells to whoever builds the DEBs, so this can be fixed in the right place.

For example the NGINX will put a initial script "nginx" in /etc/init.d/ as part of apt install.
Because the WSL v1 in windows does not have systemd, but WSL v1 can support service command and service command need the app to have initial script in /etc/init.d/ to use.

Thank you.

Good day,

I've create PR 55 #55 as a possible solution/method to allow for a /etc/default/caddy file where you specify the configfile that caddy should use, instead of editing the systemd services' file.

This is required to make use of quic-go/quic-go#3795

SO_RCVBUFFORCE (since Linux 2.6.14): Using this socket option, a privileged (CAP_NET_ADMIN) process can perform the same task as SO_RCVBUF, but the rmem_max limit can be overridden.

Otherwise the user has to manually adjust sysctl limits on his machine: https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size

The pre-configured digital ocean droplet maintained by the project does not work. It simply loads blank pages.

It also does not answer requests via https or on port 443.

Something like opkg install caddy would be really great.

Hi,
I would like to bring caddy2 to the official repository.

We have caddy2 and caddy1 in the AUR (Arch Linux User repository) already. But these source is unofficial and takes unnecessary much steps for installing caddy on Arch Linux:

https://aur.archlinux.org/packages/?O=0&K=caddy

The AUR ships these packages right now:

• caddy: This version is a recipe to build caddy v1 from source
• caddy-bin: This version ships the caddy v1 binary package (so Arch Linux users don't need to compile it)
• caddy-filemanager-standalone: This version builds caddy1 from source with only filemanager enabled.
• caddy-full-bin: This version builds caddy1 from source with all features enabled.
• caddy-git: This version builds the current caddy github/master HEAD from source
• caddy-no-telemetry: This version builds caddy with no telemetry.
• caddy-with-cgi: This version builds caddy with only cgi enabled from source.
• caddy-with-quic: This version builds caddy with only quic enabled from source.
• caddy2: This version builds caddy2 from source

You might agree with me that this situation is pretty chaotic, but most of this is not changeable, because all these packages are maintained by AUR users (not by official Arch Linux staff).
However an official caddy image in the official Arch Linux repositories would have the side effect, that some of these packages would maybe vanish or would be renamed to caddy-legacy-*.

So what are our requirements for getting caddy to Arch Linux:

1. Arch Linux is bleeding edge, therefore we would only ship the newest stable release of the newest version (this would be caddy2 as caddy1 is considered deprecated, is this correct?).
2. Arch Linux would enable all caddy2 features. We always ship our packages with all features enabled. We do this with many other packages as well. Systemd is the best example.
3. We would like to ship it without telemetry. This point is important for us, we care about users privacy. However we also have a few packages with telemetry enabled (vscode for example, although it's possible to disable telemetry via configuration).
4. We prefer signed releases, either via signed git tag/commit or via signed tarball.
5. We are working on reproducibility, so a go.mod and go.sum file is a must-have for a go project. We also prefer a simple go build -trimpath over hacky makefiles.
6. We ship our binaries with most security options enabled. Therefore we compile go binaries with go-pie (a go version that supports position independend code) and custom ldflags: LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now". Nonetheless we can do make exceptions (use go instead of go-pie, but we expect the devs to work on fixing this).

What are the next steps:

1. I would create a PKGBUILD file (this is basically a bash script), that builds caddy in a systemd-nspawn container, then I would extract the resulting binary and would test it on my system. If everything is fine, I would release it in the official Arch Linux repositories.
2. I would make a PR and add the PKGBUILD to this repository as an example. Keeping that PKGBUILD up to date here as well, could be tedious. Not sure if you expect this.
3. I would rename/delete some of the AUR packages (caddy2 would get deleted and the caddy v1 packages could get deleted by our "inactive project" rule, if you consider caddy v1 as deprecated and nobody should use it).

openSUSE Leap 15.3 was released recently^[1] so I was hoping the build target could be added to the Caddy Copr repo^[2] so it will be compatible with openSUSE Leap 15.3.

^[1] https://news.opensuse.org/2021/06/02/opensuse-leap-bridges-path-to-enterprise/
^[2] https://copr.fedorainfracloud.org/coprs/g/caddy/caddy/

@shibumi I noticed that the caddy.service file in the archlinux/ dir is outdated compared to the one in init/.

Could we update this line to reference the init/ one like ../init/caddy.service?

https://github.com/caddyserver/dist/blob/master/archlinux/PKGBUILD#L17

See #16 where we discussed the changes to the service files.

Also as an aside, we need an RC3 build! 😉

Dear Caddy team,

Request you to provide packages for aarch64 on your official Fedora/CentOS copr. Having hard time setting up caddy on aarch64 systems.

Regards,
Adarsha HD

The Caddy default page works well in Chrome. In Firefox (Firefox 124 on macOS) however, the page shows a lot of horizontal scrolling, which is not visible in Chrome.

Hi, I used the following command on CentOS 7 to install Caddy 2:
yum install yum-plugin-copr
yum copr enable @caddy/caddy
yum install caddy

But, after installation it said RC3 installed.
So, how do I install the stable version?

We're seeing an error when attempting to install Caddy using the install steps for Debian etc.

root@e04c45b8ec2e:/# curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list
# Source: Caddy
# Site: https://github.com/caddyserver/caddy
# Repository: Caddy / stable
# Description: Fast, multi-platform web server with automatic HTTPS
deb [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main
deb-src [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main
root@e04c45b8ec2e:/# apt update
Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Hit:4 http://dl.google.com/linux/chrome/deb stable InRelease
Get:5 https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease [7491 B]
Err:5 https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY ABA1F9B8875A6661
Reading package lists... Done
W: GPG error: https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY ABA1F9B8875A6661
E: The repository 'https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Is there currently an issue with the public key?

Hi, I'm not sure if this is the correct repo to report this, but v2.4.4 wasn't released on docker hub.

Receiving the following error while trying sudo apt update on debian.

E: Failed to fetch https://dl.cloudsmith.io/public/caddy/stable/deb/debian/dists/any-version/InRelease  402  Payment Required [IP: 18.67.39.68 443]
E: The repository 'https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease' is no longer signed.

Hi,

I hope this is the right project for this issue. I tried installing the latest Caddy 2.1 release, but noticed that COPR hasn't been updated since end of may (copr).

Could somebody trigger a build, so it gets available CentOS 8?

Thank you very much for Caddy!

Hello,

I have been using the Ubuntu/Debian repositories for few year now without any issues. I followed this https://caddyserver.com/docs/install#debian-ubuntu-raspbian tutorial to got everything up and running. But for a couple of days I am getting following error when running sudo apt update:

N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version InRelease' doesn't support architecture 'i386'

I am on Ubuntu 22.04.4 LTS x86_64.

caddy-stable.list looks like this:

# Source: Caddy
# Site: https://github.com/caddyserver/caddy
# Repository: Caddy / stable
# Description: Fast, multi-platform web server with automatic HTTPS


deb [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main

deb-src [signed-by=/usr/share/keyrings/caddy-stable-archive-keyring.gpg] https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main

I have tried to specify the architecture in the brakets adding "arch=amd64," but it does not work with publickey. What am I missing?

Many thanks in advance

Using Caddy from sourcesmith (deb https://dl.cloudsmith.io/public/caddy/stable/deb/debian any-version main) on Debian 10.9, I am finding it is not restarting automatically after reboot.

If I restart the machine, caddy is not running.

If I then run sudo systemctl start caddy caddy starts and runs with no errors (i.e. proving there is no fault with my configuration file or my installation)

If I then run sudo systemctl enable caddy there are no errors, but it has no effect on ensuring caddy starts after reboot.\

There seems to be some sort of race condition going on.

Immediately after reboot, I see the following in /var/log/daemon.log:

May 29 12:40:36 xyz caddy[463]: run: loading initial config: loading new config: http app module: start: tcp: listening on [my-ipv6-address]:443: listen tcp [my-ipv6-address]:443: bind: cannot assign requested address
May 29 12:40:36 xyz systemd[1]: caddy.service: Main process exited, code=exited, status=1/FAILURE
May 29 12:40:36 xyz systemd[1]: caddy.service: Failed with result 'exit-code'.

BUT I can immediately run sudo systemctl start caddy, making no configuration changes and running no prior commands, and caddy starts with no errors.

There is nothing special about my interface, its all static and hardcoded in /etc/network/interfaces no dynamic wizzardry going on.

So it seems Caddy is trying to init before the network interface has finished init (or something like that).

Caddy uses the distribution provided /etc/mime.types file to set Content-Type headers on files, however not all systems ship with /etc/mime.types by default which is packaged as part of mailcap. Caddy packages from the official repositories do not require this package currently. Without this package, files may be sent to browsers without a Content-Type header. For example out of the box without mailcap installed on RHEL 8, Caddy does not set a Content-Type for txt files.

Relevant related comments and issues:
https://discussion.fedoraproject.org/t/caddy-caddy/8578/5
caddyserver/caddy#3190
caddyserver/caddy#3959

NGINX and Apache httpd bundle their own mime.types files avoiding the distribution mime.types.
https://github.com/nginx/nginx/blob/master/conf/mime.types
https://github.com/apache/httpd/blob/trunk/docs/conf/mime.types

Caddy would ideally bundle its own list of mime types in the long run to avoid relying on distributions to provide up to date correct and relevant mime types.

I just had Caddy crash on one of my servers for the first time. Here are the logs:

Aug 04 08:12:31 caddy[712]: panic: runtime error: invalid memory address or nil pointer dereference
Aug 04 08:12:31 caddy[712]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x903750]
Aug 04 08:12:31 caddy[712]: goroutine 15427 [running]:
Aug 04 08:12:31 caddy[712]: github.com/caddyserver/certmagic.(*Config).getCertDuringHandshake(0xc0001d8680, {0x1f09a88, 0xc000138008}, _, _)
Aug 04 08:12:31 caddy[712]:         github.com/caddyserver/[email protected]/handshake.go:378 +0x1390
Aug 04 08:12:31 caddy[712]: github.com/caddyserver/certmagic.(*Config).GetCertificateWithContext(0xc0001d8680, {0x1f09a88, 0xc000138008}, 0xc0001>
Aug 04 08:12:31 caddy[712]:         github.com/caddyserver/[email protected]/handshake.go:84 +0xbff
Aug 04 08:12:31 caddy[712]: github.com/caddyserver/certmagic.(*Config).GetCertificate(0xc00040a000?, 0xc00075c120?)
Aug 04 08:12:31 caddy[712]:         github.com/caddyserver/[email protected]/handshake.go:50 +0x2a
Aug 04 08:12:31 caddy[712]: github.com/caddyserver/caddy/v2/modules/caddytls.(*ConnectionPolicy).buildStandardTLSConfig.func1(0xc0001d85b0)
Aug 04 08:12:31 caddy[712]:         github.com/caddyserver/caddy/[email protected]/modules/caddytls/connpolicy.go:232 +0x14f
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*config).getCertificate(0xc00057f380, 0xc0001d85b0)
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/common.go:1086 +0x42
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*serverHandshakeStateTLS13).pickCertificate(0xc000399be8)
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/handshake_server_tls13.go:415 +0x66
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*serverHandshakeStateTLS13).handshake(0xc000399be8)
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/handshake_server_tls13.go:60 +0x53
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*Conn).serverHandshake(0xc00003c000, {0x1f09a50, 0xc0008c2320})
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/handshake_server.go:53 +0x188
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*Conn).handshakeContext(0xc00003c000, {0x1f09af8, 0xc000758930})
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/conn.go:1540 +0x3ce
Aug 04 08:12:31 caddy[712]: github.com/quic-go/qtls-go1-20.(*Conn).HandshakeContext(0xc0000de7d0?, {0x1f09af8?, 0xc000758930?})
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/conn.go:1480 +0x25
Aug 04 08:12:31 caddy[712]: created by github.com/quic-go/qtls-go1-20.(*QUICConn).Start
Aug 04 08:12:31 caddy[712]:         github.com/quic-go/[email protected]/quic.go:179 +0xcf
Aug 04 08:12:31 systemd[1]: caddy.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Aug 04 08:12:31 systemd[1]: caddy.service: Failed with result 'exit-code'.

In order to get my apps online again I had to manually run systemctl restart caddy.service. I noticed that Caddy's service units don't instruct systemd to restart Caddy in case it crashes so I propose to make the following changes to the default systemd units:

[Unit]
StartLimitBurst=5
StartLimitIntervalSec=60
...

[Service]
Restart=on-failure
...

I do this for all of my apps in case they ever crash. It will restart them automatically up to 5 times (StartLimitBurst) within 60 seconds (StartLimitIntervalSec). If they reach the limit they transition into the failed state.

I'm hitting the same problem as outlined in caddyserver/caddy#1802. The culprit seems to be how systemd handles the LimitNProc option:

While caddy doesn't occupy that many processes, some other docker containers seem to use the same UID for their processes:

sudo ps -U caddy

    PID TTY          TIME CMD
   4491 ?        00:00:01 mailrise
  36706 ?        00:00:28 postgres
  36760 ?        00:00:01 postgres
  36761 ?        00:00:06 postgres
  36762 ?        00:00:10 postgres
  36763 ?        00:03:55 postgres
  36764 ?        00:00:14 postgres
  36765 ?        00:01:17 postgres
  36766 ?        00:00:00 postgres
1597030 ?        00:00:03 postgres
1599669 ?        00:00:03 postgres
2081581 ?        00:25:43 redis-server
2082548 ?        00:00:36 postgres
2082623 ?        00:00:34 postgres
2654461 ?        00:00:00 start.sh
2654495 ?        00:00:00 Xvfb
2654496 ?        00:00:00 dumb-init
2654497 ?        00:48:58 node
2654671 ?        00:01:16 chrome
2654672 ?        00:01:16 chrome
2654673 ?        00:01:14 chrome
2654674 ?        00:01:14 chrome
2654675 ?        00:01:16 chrome
2654676 ?        00:01:13 chrome
2654677 ?        00:01:15 chrome
2654678 ?        00:01:14 chrome
2654683 ?        00:00:00 chrome_crashpad
2654684 ?        00:00:00 chrome_crashpad
2654685 ?        00:00:00 chrome_crashpad
2654686 ?        00:00:00 chrome_crashpad
2654691 ?        00:00:00 chrome_crashpad
2654692 ?        00:00:00 chrome_crashpad
2654693 ?        00:00:00 chrome_crashpad
2654694 ?        00:00:00 chrome_crashpad
2654703 ?        00:00:00 chrome
2654704 ?        00:00:00 chrome
2654705 ?        00:00:00 chrome
2654706 ?        00:00:00 chrome
2654707 ?        00:00:00 chrome
2654708 ?        00:00:00 chrome
2654709 ?        00:00:00 chrome
2654710 ?        00:00:00 chrome
2654711 ?        00:01:14 chrome
2654712 ?        00:01:13 chrome
2654715 ?        00:00:00 chrome_crashpad
2654717 ?        00:00:00 chrome_crashpad
2654718 ?        00:00:00 chrome_crashpad
2654722 ?        00:00:00 chrome_crashpad
2654723 ?        00:00:00 chrome
2654724 ?        00:00:00 chrome
2654727 ?        00:00:00 chrome
2654728 ?        00:00:00 chrome
2654729 ?        00:00:00 nacl_helper
2654730 ?        00:00:00 nacl_helper
2654732 ?        00:00:00 chrome_crashpad
2654750 ?        00:00:00 chrome_crashpad
2654752 ?        00:00:00 chrome_crashpad
2654753 ?        00:00:00 nacl_helper
2654757 ?        00:00:00 nacl_helper
2654759 ?        00:00:00 chrome_crashpad
2654761 ?        00:00:00 chrome
2654762 ?        00:00:00 chrome
2654767 ?        00:00:00 chrome_crashpad
2654768 ?        00:00:00 chrome_crashpad
2654770 ?        00:00:00 nacl_helper
2654781 ?        00:00:00 chrome
2654786 ?        00:00:00 chrome
2654796 ?        00:00:00 chrome_crashpad
2654800 ?        00:00:00 chrome
2654802 ?        00:00:00 chrome
2654816 ?        00:00:00 chrome_crashpad
2654817 ?        00:00:16 chrome
2654818 ?        00:00:17 chrome
2654821 ?        00:00:00 chrome
2654822 ?        00:00:00 chrome
2654823 ?        00:00:17 chrome
2654824 ?        00:00:17 chrome
2654828 ?        00:00:00 nacl_helper
2654881 ?        00:00:17 chrome
2654884 ?        00:00:00 nacl_helper
2654885 ?        00:00:16 chrome
2654886 ?        00:00:17 chrome
2654901 ?        00:00:00 nacl_helper
2654907 ?        00:00:17 chrome
2654910 ?        00:00:17 chrome
2654916 ?        00:00:00 nacl_helper
2654922 ?        00:00:17 chrome
2654985 ?        00:00:19 chrome
2654999 ?        00:00:00 nacl_helper
2655029 ?        00:00:05 chrome
2655048 ?        00:00:17 chrome
2655053 ?        00:00:05 chrome
2655063 ?        00:00:16 chrome
2655065 ?        00:00:17 chrome
2655066 ?        00:00:17 chrome
2655079 ?        00:00:17 chrome
2655080 ?        00:00:16 chrome
2655085 ?        00:00:05 chrome
2655089 ?        00:00:05 chrome
2655092 ?        00:00:17 chrome
2655096 ?        00:00:05 chrome
2655097 ?        00:00:05 chrome
2655105 ?        00:00:05 chrome
2655129 ?        00:00:05 chrome
2655136 ?        00:00:05 chrome
2655179 ?        00:00:05 chrome
2655180 ?        00:00:20 chrome
2655186 ?        00:00:17 chrome
2655199 ?        00:00:05 chrome
2655223 ?        00:00:05 chrome
2655315 ?        00:00:05 chrome
2655323 ?        00:00:05 chrome
2655330 ?        00:00:05 chrome
2655337 ?        00:00:05 chrome
2655341 ?        00:00:05 chrome
2655346 ?        00:00:05 chrome
2655385 ?        00:00:05 chrome
2655391 ?        00:00:05 chrome

The systemd documentation notes that TasksMax should be preferred over LimitNProc:

Note that LimitNPROC= will limit the number of processes from one (real) UID and not the number of processes started (forked) by the service. Therefore the limit is cumulative for all processes running under the same UID. Please also note that the LimitNPROC= will not be enforced if the service is running as root (and not dropping privileges). Due to these limitations, TasksMax= (see systemd.resource-control(5)) is typically a better choice than LimitNPROC=.

https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#Process%20Properties

Hey it would be great if someone could ship the latest Caddy version to Copr. Besides other bugs, in the current Copr version (2.6.1) certs don't renew if people don't have an issuer set explicitly, which is a pretty big gotcha if people upgrade 😬.

I created automatic chocolaty package which follows your releases.

• https://chocolatey.org/packages/caddy

I will continue maintaining the package as I already have infrastructure for that job setup. The code for the package is here.

As a stable package with complete version history, you might want to mention it in the install options.

If you need me to change anything, rise a ticket on repository mentioned above.

i would like to suggest adding a snap container as a official install mode for Caddy.

The current rpm and deb packaging specs rely on the bit-rot completion scripts which used a tool called completely. The PR caddyserver/caddy#4565 adds the benefit of adding 2 commands to generate manpages and completions scripts for {bash,fish,zsh}. Once the PR lands, the packaging specs should be updated to use the commands to generate fresh and accurate completion scripts and manpages.

I'm trying to install Caddy on Raspbian 11 (Bullseye) and I'm getting the following error:

E: Unable to locate package caddy

I've followed the directions on the docs and added the caddy repository to apt.

I've also noticed that if you go to https://dl.cloudsmith.io/public/caddy/stable/deb/debian (the URL from the install section of the docs) it's returning a 404 right now. Is there an issue distributing the package?

I've started getting the following...

sudo service caddy status
● caddy.service - Caddy
   Loaded: loaded (/lib/systemd/system/caddy.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2023-06-25 07:32:58 BST; 1h 18min ago
     Docs: https://caddyserver.com/docs/
  Process: 2096 ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile (code=exited, status=2)
 Main PID: 2096 (code=exited, status=2)

Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]:         net/net.go:183 +0x45 fp=0xc0048dda98 sp=0xc0048dda50 pc=0x55b905
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]: net.(*TCPConn).Read(0xc0048ddb30?, {0xc0018d86c0?, 0xc00121f7a0?, 0x18?})
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]:         <autogenerated>:1 +0x29 fp=0xc0048ddac8 sp=0xc0048dda98 pc=0x56e029
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]: crypto/tls.(*atLeastReader).Read(0xc00121f7a0, {0xc0018d86c0?, 0xc00121f7a0?, 0x0?})
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]:         crypto/tls/conn.go:788 +0x3d fp=0xc0048ddb10 sp=0xc0048ddac8 pc=0x6c369d
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]: bytes.(*Buffer).ReadFrom(0xc002ac5790, {0x1ddc440, 0xc00121f7a0})
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]:         bytes/buffer.go:202 +0x98 fp=0xc0048ddb68 sp=0xc0048ddb10 pc=0x510798
Jun 25 07:32:52 www.sitenameremoved.co.uk caddy[2096]: crypto/tls.(*Conn).readFromUntil(0xc002ac5500, {0x1de1e60?, 0xc000624280}, 0x0?)
Jun 25 07:32:58 www.sitenameremoved.co.uk systemd[1]: caddy.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 25 07:32:58 www.sitenameremoved.co.uk systemd[1]: caddy.service: Failed with result 'exit-code'.

server is running
v2.6.4 h1:2hwYqiRwk1tf3VruhMpLcYTg+11fCdr8S3jhNAdnPy8=

Any advice on how I can debug?

Hi,

First of all, thanks for the great work on caddy.

I would like to start by saying that the current packaging solution is really appreciated and solves the problem of most users, and this issue aims in proposing a solution to also bring the work in packaging to the users building custom binaries.

My experience in packaging software is limited, the intention of this proposal is to bring the case to the attention of the maintainers that have more experience than me.

On the community discord we had a case where the user had some issues deploying a custom-built caddy to his server and the error messages sent me down the wrong rabbit hole which triggered some discussion on how to avoid such a thing to happen again and to make the experience more pleasant for the users building custom binaries.

I did test some of the options below, except for the one splitting caddy into multiple packages.

Problem statement:

Any user that utilizes xcaddy to build custom binaries, will also be forced into a position that they need to manage other operational system configurations by themselves, like for example the default configuration and systemd service definition.

One can states that any user utilizing xcaddy should be considered an advanced user, and this should not be a problem, but there are some cases where this can cause trouble even for advanced users to troubleshoot.

For example ReadWriteDirectories configuration on systemd, if any directory is missing, caddy will throw a 'read-only file system' error instead of an access denied or even a more descriptive error, which usually sends into a completely different troubleshooting path.

But considering how much work and thought was put into xcaddy to make the experience pleasant and seamless to users, I assume that this is something that would be welcomed to be discussed.

Options:

There are multiple options on how to solve this issue and provide a better experience for users that want to build custom binaries.

Just let users install the package and replace binaries

xcaddy build ...
cp ./caddy /usr/bin/caddy

Pros:

• Simplest possible solution

Cons:

• This will break package upgrades
• It can trigger alerts for HIDS that uses package checksums
• It's not the standard way to handle such configuration on Debian based distributions

dpkg-divert

dpkg-divert --divert /usr/bin/caddy.default --rename /usr/bin/caddy
xcaddy build ...
cp ./caddy /usr/bin/caddy

Pros:

• Does not require any changes from package maintainers

Cons:

• Requires user knowledge and documentation
• It's not the standard procedure for other packages, it's considered more of a workaround.

update-alternatives

xcaddy build ...
cp ./caddy /usr/bin/caddy.custom
update-alternatives --install /usr/bin/caddy caddy /usr/bin/caddy.custom 10

Pros:

• It's a behavior that is a little bit more known and expected
• It allows users to easily move back and forth between binaries

Cons:

• It's a more common pattern when multiple packages that contains the same binary, like for example vim.tiny, and not for custom binaries

break caddy into two different packages, caddy and caddy-common

caddy would bring the default binary, and caddy-common just other distribution files, including systemd and default configuration.

apt install caddy-common
xcaddy build ...
cp caddy /usr/bin/caddy

Pros:

• It's simple and does not require much work from users, which will reduce human errors

Cons:

• This breaks backward compatibility with packages
• It requires more work from the packaging side

One point to note is that to avoid breaking backward compatibility, we can provide 3 packages, being caddy a transitional meta-package that depends on both caddy-default and caddy-common this way we offer an upgrade path for users with older package versions.

Requesting an script that can be used to automatically detect and download the basic static binary for the target system and report the system specs.

My OpenWRT router reports "mips" when I uname -m but further inspection it is mipsle and I want to be able to try to replace its web server with caddy as its easier to manager