Giter VIP home page Giter VIP logo

mort's Introduction

Mort

Tests Codecov Go Report Card Godoc Releases LICENSE

An S3-compatible image processing server written in Go. Still in active development.

Features

  • HTTP server
  • Resize, Rotate, SmartCrop
  • Convert (JPEG, PNG , BMP, Webp)
  • Multiple storage backends (disk, S3, http)
  • Fully modular
  • S3 API for listing and uploading files
  • Requests collapsing
  • Build in rate limiter
  • HTTP Range and Conditional requests
  • Compression (gzip, brotli)
  • Env variables driven configuration
  • Write you own transform parser using tengo
  • cloudinary image transformation and upload

And more see changelog for more info

Demo


Original image

Click on result image to see URL. More examples can be found in Image Operations list

Description Result (to see result click on image)

preset: small

(preserve aspect ratio) width: 75

preset: blur

  • resize image (preserve aspect ratio) width: 700

  • blur image with sigma 5.0

preset: webp

  • resize image (preserve aspect ratio) width: 1000

  • and change format to webp

preset: watermark

  • resize image (preserve aspect ratio) width: 1300

  • and add watermark

Usage

Mort can be used directly from the Internet and behind any proxy.

Install

go get github.com/aldor007/mort/cmd/

Command line help

$ ./mort
Usage of  mort
  -config string
    	Path to configuration (default "/etc/mort/mort.yml")

Configuration

Example configuration used for providing demo images:

headers: #  add or overwrite all response headers of given status. This field is optional
  - statusCodes: [200]
    values:
      "cache-control": "max-age=84000, public"

buckets: # list of available buckets
    demo:    # bucket name
        keys: # list of S3 keys (optional)
          - accessKey: "access"
            secretAccessKey: "random"
        transform: # config for transforms
            path: "\\/(?P<presetName>[a-z0-9_]+)\\/(?P<parent>[a-z0-9-\\.]+)" # regexp for transform path
            kind: "presets-query" #  type of transform or "query"
            presets: # list of presets
                small:
                    quality: 75
                    filters:
                        thumbnail:
                            width: 150
                blur:
                    quality: 80
                    filters:
                        thumbnail:
                            width: 700
                        blur:
                          sigma: 5.0
                webp:
                    quality: 100
                    format: webp
                    filters:
                        thumbnail:
                            width: 1000
                watermark:
                    quality: 100
                    filters:
                        thumbnail:
                            width: 1300
                        watermark:
                            image: "https://i.imgur.com/uomkVIL.png"
                            position: "top-left"
                            opacity: 0.5
                smartcrop:
                    quality: 80
                    filters:
                      crop:
                        width: 200
                        height: 200
        storages:
             basic: # retrieve originals from s3
                 kind: "s3"
                 accessKey: "acc"
                 secretAccessKey: "sec"
                 region: ""
                 endpoint: "http://localhost:8080"
             transform: # and store it on disk
                 kind: "local-meta"
                 rootPath: "/var/www/domain/"
                 pathPrefix: "transform"

List of all image operations can be found in Image-Operations.md

More details about configuration can be found in Configuration.md

Write you own URL parser

Docker

See Dockerfile for image details.

Pull docker image

docker pull ghcr.io/aldor007/mort

Create you custom docker deployment

Create Dockerfile or use Dockerfile.service

# Dockerfile.service 
FROM ghcr.io/aldor007/mort:latest

# add your config
ADD config.yml /etc/mort/mort.yml

Build container

docker build -f Dockerfile.service -t myusername/mort

Run docker

docker run -p 8080:8080 myusername/mort

Full example you can find here

Development

  1. Make sure you have a Go language compiler >= 1.9 (required) and git installed.
  2. Install libvips like described on bimg page
  3. Ensure your GOPATH is properly set.
  4. Download it
git clone  https://github.com/aldor007/mort.git $GOPATH/src/github.com/aldor007/mort
cd $GOPATH/src/github.com/aldor007/mort
  1. Install dependencies:
go mod vendor

Run unit tests:

make unit

Run integration tests:

make integrations

If you encounter following problem go build gopkg.in/h2non/bimg.v1: invalid flag in pkg-config --cflags: -Xpreprocessor while running tests in your IDE add this to your environment:

export CGO_CFLAGS_ALLOW="-Xpreprocessor"

Run tests using docker image

make docker-tests

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Inspirations

mort's People

Contributors

aldor007 avatar dependabot[bot] avatar mkrauser avatar semantic-release-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mort's Issues

同学,您这个项目引入了468个开源组件,存在5个漏洞,辛苦升级一下

检测到 aldor007/mort 一共引入了468个开源组件,存在5个漏洞

漏洞标题:jwt-go 安全漏洞
缺陷组件:github.com/dgrijalva/[email protected]+incompatible
漏洞编号:CVE-2020-26160
漏洞描述:jwt-go是个人开发者的一个Go语言的JWT实现。
jwt-go 4.0.0-preview1之前版本存在安全漏洞。攻击者可利用该漏洞在使用[]string{} for m[\"aud\"](规范允许)的情况下绕过预期的访问限制。
影响范围:(∞, 4.0.0-preview1)
最小修复版本:4.0.0-preview1
缺陷组件引入路径:main@->github.com/dgrijalva/[email protected]+incompatible

另外还有5个漏洞,详细报告:https://mofeisec.com/jr?p=a2f25d

Singing query transformation

Add new parameter sig that will be required for validation. Algorithm for singing transformation should be easy to implement on front

help on configuration

I want to setup Mort to serve webp images on Wordpress on-the-fly.
i.e.
BEFORE: https://mydomain.com/wp-content/filename.jpg (content type: jpeg)
AFTER: https://mydomain.com/wp-content/filename.jpg (content type: webp)

My setup is:
Cloudflare -> Docker

  • Wordpress (with Apache)
  • Mort

Apache ProxyPass passing image request to Mort

ProxyPassMatch "^/(.*\.jpg)$" "http://mort:8080/$1"
ProxyPassMatch "^/(.*\.jpeg)$" "http://mort:8080/$1"
ProxyPassMatch "^/(.*\.png)$" "http://mort:8080/$1"

On Docker-Compose wp-content is mounted onto Mort container:

version: '3.8'
services:
  mort:
    container_name: mort
    image: ghcr.io/aldor007/mort
    volumes:
      - type: bind
        source: ./misc/mort-config.yml
        target: /etc/mort/mort.yml
      - type: bind
        source: ./wordpress
        target: /root
    restart: unless-stopped

Hence

# ls -la /root
total 28
drwxrwxrwx  3     1000     1000 4096 Aug 11 04:34 .
drwxr-xr-x  1 root     root     4096 Aug 12 02:49 ..
-rwxrwxrwx  1     1000     1000   34 May 16 02:13 .git
-rwxrwxrwx  1     1000     1000 1009 Aug 11 05:43 .gitignore
-rwxrwxrwx  1     1000     1000 5480 May 16 02:13 wp-config-docker.php
drwxr-xr-x 14 www-data www-data 4096 Aug 12 03:14 wp-content

This is configuration of Mort:

#server:
#  listen: "0.0.0.0:8080"
#  monitoring: ""
#  cache:
#    type: "memory"
#    cacheSize: 50000
#    maxCacheItemSizeMB: 50
#  requestTimeout: 70
#  internalListen: "0.0.0.0:8081"
#  plugins:
#    - "webp"

headers:
  - statusCodes: [200]
    values:
      "cache-control": "max-age=84000, public"
  - statusCodes: [404, 400]
    values:
      "cache-control": "max-age=60, public"
  - statusCodes: [500, 503]
    values:
      "cache-control": "max-age=10, public"

buckets:
  demo:
    transform:
      path: "\\/(?P<presetName>[a-z0-9_]+)\\/(?P<parent>.*)"
      kind: "presets"
      parentStorage: "mounted-dir"
      presets:
        wp-content:
          quality: 100
          format: webp
          filters:
            thumbnail:
              width: 1000
    storages:
      basic:
        kind: "local-meta"
        rootPath: "/root"
      mounted-dir:
        kind: "local-meta"
        rootPath: "/root"

docker logs -f mort shows this error:

{"level":"warn","ts":"2022-08-12T03:14:45.581Z","caller":"middleware/s3.go:179","msg":"S3Auth no bucket for access key","hostname":"0e2d7b3c2416","pid":1}

May I know if how should I configure it?
Thanks.

Add support for start/end clipping

Not sure if this is in there yet, but the ability to specify the start and end points, preferably multiple of them per video so you can clip out smaller bits of video would be very nice to have.

panic: memory lock

routine 164 [running]:\nruntime/debug.Stack()\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x65\ngithub.com/go-chi/chi/v5/middleware.Recoverer.func1.1()\n\t/github/home/go/pkg/mod/github.com/go-chi/chi/[email protected]/middleware/recoverer.go:29 +0xa9\npanic({0x11a3400, 0x1e72960})\n\t/usr/local/go/src/runtime/panic.go:838 +0x207\ngithub.com/djherbis/stream.(*Reader).Read(0x453432?, {0xc0007be000?, 0x111e480?, 0xc000708801?})\n\t/github/home/go/pkg/mod/github.com/djherbis/[email protected]/reader.go:32 +0x4b\nio.copyBuffer({0x1642c80, 0xc000708840}, {0x7fcbb4283608, 0xc000051e40}, {0x0, 0x0, 0x0})\n\t/usr/local/go/src/io/io.go:426 +0x1b2\nio.Copy(...)\n\t/usr/local/go/src/io/io.go:385\ngithub.com/go-chi/chi/v5/middleware.(*httpFancyWriter).ReadFrom(0xc000708840, {0x7fcbb4283608?, 0xc000051e40?})\n\t/github/home/go/pkg/mod/github.com/go-chi/chi/[email protected]/middleware/wrap_writer.go:189 +0x4c\nio.copyBuffer({0x7fcbb420a7d8, 0xc000708840}, {0x7fcbb4283608, 0xc000051e40}, {0x0, 0x0, 0x0})\n\t/usr/local/go/src/io/io.go:412 +0x14b\nio.Copy(...)\n\t/usr/local/go/src/io/io.go:385\ngithub.com/aldor007/mort/pkg/response.(*Response).Send(0xc0003e1bc0, {0x7fcbb42835d8, 0xc000708840})\n\t/__w/mort/mort/pkg/response/response.go:255 +0x4b3\ngithub.com/aldor007/mort/pkg/response.(*Response).SendContent(0x126afa0?, 0xc0002ef770?, {0x7fcbb42835d8?, 0xc000708840?})\n\t/__w/mort/mort/pkg/response/response.go:267 +0x136\nmain.main.func1.1({0x7fcbb42835d8, 0xc000708840}, 0xc0000e4400)\n\t/__w/mort/mort/cmd/mort/mort.go:277 +0xaf8\nnet/http.HandlerFunc.ServeHTTP(0xc0005ae1b0?, {0x7fcbb42835d8?, 0xc000708840?}, 0xc000158214?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/aldor007/mort/pkg/middleware.(*S3Auth).Handler.func1({0x7fcbb42835d8, 0xc000708840}, 0xc0000e4400)\n\t/__w/mort/mort/pkg/middleware/s3.go:74 +0x764\nnet/http.HandlerFunc.ServeHTTP(0x9c458d4482de76a6?, {0x7fcbb42835d8?, 0xc000708840?}, 0x0?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/aldor007/mort/pkg/object/cloudinary.UploadInterceptorMiddleware.Handler.func1({0x7fcbb42835d8?, 0xc000708840?}, 0xc00063f7a0?)\n\t/__w/mort/mort/pkg/object/cloudinary/upload_interceptor_middleware.go:106 +0x6a\nnet/http.HandlerFunc.ServeHTTP(0x12ef2c0?, {0x7fcbb42835d8?, 0xc000708840?}, 0x97668f?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/go-chi/chi/v5/middleware.Recoverer.func1({0x7fcbb42835d8?, 0xc000708840?}, 0xc0005ae400?)\n\t/github/home/go/pkg/mod/github.com/go-chi/chi/[email protected]/middleware/recoverer.go:38 +0x83\nnet/http.HandlerFunc.ServeHTTP(0xc0000e4300?, {0x7fcbb42835d8?, 0xc000708840?}, 0x203000?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/go-chi/httplog.Handler.func1.1({0x164c598, 0xc000198000}, 0xc0000e4300)\n\t/github/home/go/pkg/mod/github.com/go-chi/[email protected]/httplog.go:63 +0x245\nnet/http.HandlerFunc.ServeHTTP(0x164d0d8?, {0x164c598?, 0xc000198000?}, 0x163ea08?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/go-chi/chi/v5/middleware.RequestID.func1({0x164c598, 0xc000198000}, 0xc0000e4100)\n\t/github/home/go/pkg/mod/github.com/go-chi/chi/[email protected]/middleware/request_id.go:76 +0x354\nnet/http.HandlerFunc.ServeHTTP(0x413387?, {0x164c598?, 0xc000198000?}, 0x1e72401?)\n\t/usr/local/go/src/net/http/server.go:2084 +0x2f\ngithub.com/go-chi/chi/v5.(*ChainHandler).ServeHTTP(0x164d030?, {0x164c598?, 0xc000198000?}, 0x1e724c0?)\n\t/github/home/go/pkg/mod/github.com/go-chi/chi/[email protected]/chain.go:31 +0x2c\ngithub.c

Add new lock strategy that allow run mort in multi instance mode - RedisLock

If we like to run mort in a multi-instance mode it should have an opportunity to acquiring a lock that is shared between instances. For purpose of such locking most suitable is redis.
Tasks:

  • implement redis lock (save only buffered response in redis)
  • add configuration that allows user choose lock strategy (Memory, Nop, Redis)
  • update doc
  • write tests

Fail to get Images from S3

Hey,

first of all, thank you very much for your work.
Here's my Problem: I've configured a preset bucket like this:

buckets:
    media:
        keys:
          - accessKey: "..."
            aecretAccessKey: "..."
        transform:
            path: "\\/(?P<presetName>[a-z0-9_]+)\\/(?P<parent>.*)"
            checkParent: true
            kind: "presets"
            presets:
                ...
        storages:
            basic:
                kind: "http"
                url: "https://some-server.de/images<item>"

I can successfully access http://localhost:8080/media/test-image.jpg, but http://localhost:8080/media/test/image.jpg is not working.
https://some-server.de/images/test-image.jpg and https://some-server.de/images/test/image.jpg both exist.

When I change the storage-backend to a S3-Bucket, the result is the same.

The log only shows "FileObject decodePreset unknown preset" for the file which cannot be accessed.

I think I tried everything configuration wise. I hope you can help me out here!

Thanks again for your work!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.