Giter VIP home page Giter VIP logo

webp_server_go's Introduction

This is a Server based on Golang, which allows you to serve WebP images on the fly. It will convert jpg,jpeg,png files by default, this can be customized by editing the config.json..

  • currently supported image format: JPEG, PNG, BMP, GIF(static image for now)

e.g When you visit https://a.com/1.jpg,it will serve as image/webp without changing the URL.

For Safari and Opera users, the original image will be used.

General Usage Steps

1. Download or build the binary

Download the webp-server from release page.

Wanna build your own binary? Check out build section

2. Dump config file

./webp-server -dump-config > config.json

The default config.json may look like this.

{
	"HOST": "127.0.0.1",
	"PORT": "3333",
	"QUALITY": "80",
	"IMG_PATH": "/path/to/pics",
	"EXHAUST_PATH": "/path/to/exhaust",
	"ALLOWED_TYPES": ["jpg","png","jpeg"]
}

Regarding the IMG_PATH section in config.json. If you are serving images at https://example.com/pics/tsuki.jpg and your files are at /var/www/image/pics/tsuki.jpg, then IMG_PATH shall be /var/www/image.

EXHAUST_PATH is cache folder for output webp images, with EXHAUST_PATH set to /var/cache/webp in the example above, your webp image will be saved at /var/cache/webp/pics/tsuki.jpg.1582558990.webp.

3. Run

./webp-server --help
Usage of ./webp-server:
  -child
        is child process
  -config string
        /path/to/config.json. (Default: ./config.json) (default "config.json")
  -dump-config
        Print sample config.json
  -dump-systemd
        Print sample systemd service file.
  -jobs int
        Prefetch thread, default is all. (default 8)
  -prefetch
        Prefetch and convert image to webp
  -prefork
        use prefork

Prefetch

Prefetch will convert all your images to webp. Don't worry, WebP Server will start, you don't have to wait until prefetch completes.

./webp-server -prefetch

If you want to control threads to use while prefetching, add -jobs=4. By default, it will utilize all your CPU cores.

# use 4 cores
./webp-server -prefetch -jobs=4

dump systemd service file

The standard systemd service file will show on your screen. You many want to use > to redirect to a file.

./webp-server -dump-systemd

screen or tmux

Use screen or tmux to avoid being terminated. Let's take screen for example

screen -S webp
./webp-server --config /path/to/config.json

(Use Ctrl-A-D to detach the screen with webp-server running.)

systemd

Don't worry, we've got you covered!

Download webp-server to /opt/webps/webp-server, and create a config file to /opt/webps/config.json, then,

./webp-server -dump-systemd > /lib/systemd/system/webp-server.service
systemctl daemon-reload
systemctl enable webp-server.service
systemctl start webp-server.service

docker

We've build docker images on hub.docker.com. If you want to run webp-server insider docker container, you can run the command below,

docker run -d -p 3333:3333 -v /path/to/pics:/opt/pics --name webps webpsh/webps

The path path/to/pics is your images serving in local. The path /opt/pics unable modify because it define in the config.json when building docker image. The cache folder of EXHAUST_PATH default define in /opt/exhaust , you can also mount the local dir for the cache folder by using -v /path/to/exhaust:/opt/exhaust option.

4. Nginx proxy_pass

Let Nginx to proxy_pass http://localhost:3333/;, and your webp-server is on-the-fly.

CDN Problem

If you use CDN(such as Cloudflare) for your website, we'd recommend you to add a no-cache header like the example below, this will prevent your images from being cached and webp images rendered to Safari users(which will of course cause some troubles).

Nginx example

This is an example for a typical Wordpress installation.

location ^~ /wp-content/uploads/ {
      add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
      proxy_pass http://127.0.0.1:3333;
}

If you use Caddy, you may refer to 优雅的让 Halo 支持 webp 图片输出.

Auto update

This tool will check for new release whenever you run it. The updated binary will be save to update dir.

Build your own binaries

Install latest version of golang, enable go module, clone the repo, and then...

make

Due to the limitations of webp module, you can't cross compile this tool. But the binary will work instantly on your platform and arch

Size

  • webp_server with node_modules: 43M
  • webp-server(go) single binary: 15M

Performance

It's basically between ExpressJS and Fiber, much faster than the http package of course.

Convenience

  • webp_server: Clone the repo -> npm install -> run with pm2
  • webp-server(go): Download a single binary -> Run

TODO

  • This version doesn't support header-based-output, which means Safari users will not see the converted webp images, this should be fixed in later releases.
  • Multi platform support.
  • A better way to supervise the program.
  • Get rid of render-blocking effect on first render.
  • Prefetch on server initialization.
  • Custom exhaust path.
  • Multiple listen address.

Benchmark on convert

8 core

file_size_range file_num src_size dist_size total user system cpu core
(10KB,500KB) 4600 1.3G 310M 1:44.49 905.41s 9.55s 875% 8
(500KB,1MB) 3500 2.4G 361M 2:04.81 1092.50s 7.98s 881% 8
(1MB,4MB) 2000 3.8G 342M 2:32.64 1345.73s 10.84s 888% 8
(4MB,32MB) 500 3.6G 246M 1:44.53 916.91s 12.03s 888% 8

1,2,4,8 core

file_size_range file_num src_size dist_size 8 4 2 1
(10KB,500KB) 4600 1.3G 310M 1:44.49 2:18.49 3:36.05 5:20.88
(500KB,1MB) 3500 2.4G 361M 2:04.81 2:49.46 4:16.41 6:28.97
(1MB,4MB) 2000 3.8G 342M 2:32.64 3:26.18 5:22.15 7:53.45
(4MB,32MB) 500 3.6G 246M 1:44.53 2:21.22 3:39.16 5:28.65

Related Articles(In chronological order)

License

WebP Server is under the GPLv3. See the LICENSE file for details.

webp_server_go's People

Contributors

n0vad3v avatar bennythink avatar bluecocoa avatar knifec avatar muzi502 avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.