alibaba / nginx-http-concat Goto Github PK

A Nginx module for concatenating files in a given context: CSS and JS files usually

C 100.00%

nginx-http-concat's Introduction

HTTP Concatenation module for Nginx

Introduction

This is a module that is distributed with tengine which is a distribution of Nginx that is used by the e-commerce/auction site Taobao.com. This distribution contains some modules that are new on the Nginx scene. The ngx_http_concat module is one of them.

The module is inspired by Apache's modconcat. It follows the same pattern for enabling the concatenation. It uses two ?, like this:

http://example.com/??style1.css,style2.css,foo/style3.css

If a third ? is present it's treated as version string. Like this:

http://example.com/??style1.css,style2.css,foo/style3.css?v=102234

Configuration example

location /static/css/ {
    concat on;
    concat_max_files 20;
}
    
location /static/js/ {
    concat on;
    concat_max_files 30;
}

Module directives

concat on | off

default: concat off

context: http, server, location

It enables the concatenation in a given context.

concat_types MIME types

default: concat_types: text/css application/x-javascript

context: http, server, location

Defines the MIME types which can be concatenated in a given context.

concat_unique on | off

default: concat_unique on

context: http, server, location

Defines if only files of a given MIME type can concatenated or if several MIME types can be concatenated. For example if set to off then in a given context you can concatenate Javascript and CSS files.

Note that the default value is on, meaning that only files with same MIME type are concatenated in a given context. So if you have CSS and JS you cannot do something like this:

http://example.com/static/??foo.css,bar/foobaz.js

In order to do that you must set concat_unique off. This applies to any other type of files that you decide to concatenate by adding the respective MIME type via concat_types,

concat_max_files numberp

default: concat_max_files 10

context: http, server, location

Defines the maximum number of files that can be concatenated in a given context. Note that a given URI cannot be bigger than the page size of your platform. On Linux you can get the page size issuing:

getconf PAGESIZE

Usually is 4k. So if you try to concatenate a lot of files together in a given context you might hit this barrier. To overcome that OS defined limitation you must use the large_client_header_buffers directive. Set it to the value you need.

concat_delimiter: string

default: NONE

context: http, server, locatione

Defines the delimiter between two files. If the config is concat_delimiter "\n",a '\n' would be inserted betwen 1.js and 2.js when visted http://example.com/??1.js,2.js

concat_ignore_file_error: on | off

default: off

context: http, server, location

Whether to ignore 404 and 403 or not.

Installation

Clone the git repo.

git clone git://github.com/alibaba/nginx-http-concat.git

Add the module to the build configuration by adding --add-module=/path/to/nginx-http-concat.
Build the nginx binary.
Install the nginx binary.
Configure contexts where concat is enabled.
Build your links such that the above format, i.e., all URIs that have files that are to be concatenated have a ?? prefix. The HTML produced would have something like this inside the <head> element for concatenating CSS files.
```
<link rel="stylesheet" href="??foo1.css,foo2.css,subdir/foo3.css?v=2345" />
```
Similarly for Javascript files you should have:
```
<script src="??bar1.js,bar22.css,subdir/bar3.js?v=3245" />
```
Now if you open up the network tab on firebug or on safari/chrome/chromium browser inspector you should see a single bar where before there were many. Congratulations you're now using file concatenation at the server level. No longer messing around with scripts for aggregating files. Note although that there's no minification of files. So you might want to minify the files before concatenating them.
Done.

Tagging releases

Perusio is maintaing a tagged release at http://github.com/alibaba/nginx-http-concat in synch with the Tengine releases. Refer there for the latest uncommitted tags.

Other tengine modules on Github

footer filter: allows to add some extra data (markup or not) at the end of a request body. It's pratical for things like adding time stamps or other miscellaneous stuff without having to tweak your application.
http slice: allows to serve a file by slices. A sort of reverse byte-range. Useful for serving large files while not hogging the network.

Other builds

As referred at the outset this module is part of the tengine Nginx distribution. So you might want to save yourself some work and just build it from scratch using tengine in lieu if the official Nginx source.
If you fancy a bleeding edge Nginx package (from the dev releases) for Debian made to measure then you might be interested in Perusio's HA/HP debian Nginx package with built-in support for nginx-http-concat. Instructions for using the repository and making the package live happily inside a stable distribution installation are provided.

Acknowledgments

Thanks to Joshua Zhu and the Taobao platform engineering team for releasing tengine. Also for being kind enough to clarify things regarding this module on the Nginx mailing list.

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

nginx-http-concat's People

Contributors

Stargazers

Watchers

Forkers

aimicheng zhuzhaoyuan zhangjiejie dinic willis x-code-x jjkkwwww starsjz skysbird patbiti fyatao liusongsen linkul chenbk85 twlkyao mysoko jiangzhonghui kevgithub cosmocommerce dwoon gaohf troland kirinse ryanwli nginx-modules zj1990 fenglvming wujiy rzs840707 sangyun csdnnet kinsen limingziqiang achun2080 ccvcd luxueyan yiliaofan happy10086 jefth ascii26 adomaxbd buildsufai reedf grandi23 gitbbrid zhangxuekui sobeautiy shmnhdouble2013 hanwangkun liqingrikeiikyeong cluo renming xudianyang kooooooma richarwu googlezhang vanyoo soulteary poneyo iseeyo cloudxtreme daoyuly leeomar antife-yinyue kunnet webx32 tdp100 magicento iloveyou416068 jacklei-china jimmyjzhang lockgit dolymood is00hcw gavinjx nimoc huangjialin liu1993s yoggolang finderl hinet fenglang0203 wuyumin sixgodx zhouguoqing917 yaoboqin heiyou ansonchen xiaoking yuxianhua guoxiaod scottk212 michealjou mboo2005 ckmilse josh-techindeed lihuibin xty88645 alertisme petertonoli

nginx-http-concat's Issues

问一下nginx是通过homebrew安装的，怎么装这个模块？

R.T.

unknown directive "concat" in /etc/nginx/vhosts/www.test.com:10

在ubuntu 中使用时候报的错误， www,test,com 是一个站点的配置，如：
server{
listen 80;
server_name www.test.com;

    access_log /home/xiechao/test/test.log;
    location / {
        root  /home/xiechao/test/;
        index index.html;
    }
    location /css/ {concat on;}

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}

能支持 nginx 的 add-dynamic-module 安装吗？

1.9 以后的nginx 已经支持动态加载module 了，不知道 concat 能否跟进一下

root目前只支持绝对地址吗？

如下：

location /img/ {
  root ../images;
  concat on;
  concat_max_files 20;
}

这样配置下，只能找到普通静态请求，二合并静态资源请求都不行
是concat不支持吗？

nginx-http-concat是否支持Https

该模块不支持nginx-1.12?

我的ngixn版本是当前最新稳定版1.12 按照文档要求安装和配置以后，能正常访问，但是请求没有合并。我看到当前nginx-http-concat最近一次提交时间是2013年，该模块支持的最新的nginx版本是哪个？

对js中的注释代码块的解析能力不好

/*! AdminLTE app.js
 * ================
 * Main JS application file for AdminLTE v2. This file
 * should be included in all pages. It controls some layout
 * options and implements exclusive AdminLTE plugins.
 *
 * @Author  Almsaeed Studio
 * @Support <http://www.almsaeedstudio.com>
 * @Email   <[email protected]>
 * @version 2.3.1
 * @license MIT <http://opensource.org/licenses/MIT>
 */

/*! DataTables 1.10.7
 * ©2008-2015 SpryMedia Ltd - datatables.net/license
 */

这两个注释会出错，我把这两个注释去掉的话，就正常了。

在合并多个文件的容错

在合并多个文件当其中一个js或css找不到的时候，不要提示404，可以忽略

第一次用nginx_http_concat 发现了一些问题

css中图片路径相对路径找不到
哪些文件用此类方法比较好是公共文件吗

怎么知道nginx-http-concat模块编译进了nginx

由于我的服务器(Linux)之前已经安装一个nginx，然后这次我把nginx的功能加入进来，但是我之前在我的Mac电脑配置的完全工作。但是在我的服务器上相同的配置就是不工作。所以我在想是不是nginx没有编译进入二进制文件啊。

./configure和make的时候都可以搜索到nginx-http-concat正确配置了。
make install里面并没有发现什么nginx-http-concat的关键字

所以我的问题就是：如何可以确定我的nginx-http-concat模块编译进了我的nginx了？

如何可以确定我的nginx-http-concat模块编译进了我的nginx了？

./configure和make的时候都可以搜索到nginx-http-concat正确配置了。
make install里面并没有发现什么nginx-http-concat的关键字

所以我的问题就是：如何可以确定我的nginx-http-concat模块编译进了我的nginx了？

Nginx 1.6.2 合并js，css都404 - -求救

location /img/ {
concat on;
concat_max_files 20;
}

话说这个/img/是域名相对地址，还是主机本地相对地址，我本地相对地址是/web/img/

反正改过，两者都没成功。

不知道这边还有没有人维护呢？

求助，试用了这个模块合并后的请求，浏览器一直返回400错误，文件类型是一样的，都是js，具体设置如下：

nginx 的版本：
mactekiMacBook-Pro-3:~ mac$ nginx -V
nginx version: nginx/1.5.7
built by clang 4.2 (clang-425.0.28) (based on LLVM 3.2svn)
TLS SNI support enabled
configure arguments: --prefix=/usr/local/nginx --user=www --group=www --with-http_ssl_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_gzip_static_module --with-http_stub_status_module --http-proxy-temp-path=/var/nginx/proxy --http-fastcgi-temp-path=/var/nginx/fastcgi --http-client-body-temp-path=/var/nginx/client --add-module=nginx-http-concat

nginx.conf设置：
location /sea-modules/ {
concat on;
concat_max_files 20;
}
文件目录就是html-->index+sea-modules

浏览器报错提示：
Request URL:http://localhost/sea-modules/arale/??class/1.1.0/class.js,events/1.1.0/events.js
Request Method:GET
Status Code:400 Bad Request

紧急求助，还望各位大牛能帮我解答一下

那些情况下适用？如果适用了cdn有没有必要做合并？

那些情况下适用？如果适用了cdn有没有必要做合并？是不是需要看具体的情况？

nginx-http-concat 和 nginx反向代理 proxy_pass 能否兼容？

nginx-http-concat 和 nginx反向代理 proxy_pass 能否兼容？
试了一下，貌似不行啊？
怎么解决呢？
配置如下：

`location /
{
proxy_pass http://127.0.0.1:8083;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 60;
proxy_read_timeout 600;
proxy_send_timeout 600;

        concat on;
        concat_max_files 20;
        concat_unique off;
        concat_ignore_file_error on;
    }`

??的形式可否支持自定义呢

403 Forbidden错误

    location /frontend {
        concat on;
        concat_max_files 20;
    concat_ignore_file_error off;
}

http://localhost:9080/frontend/??job.js,news.js

请问是什么原因啊，我看了下，nginx-http-concat已经编辑成功了

Concatenating upstream responses

Hi, I'd like to concatenate some upstream responses when nginx is configured as a reverse proxy server. Something like this:

location / {
  concat on;
  proxy_pass http://upstream:3000/;
}

Is this possible?

css合并后有个小红点，导致部分样式失效情况

css合并后在前后文件连接点处有个小红点（在chrome下可以看到，生成的代码看上去并无异常），导致部分样式不生效，文件编码为utf-8格式，求大神指点

nginx: [emerg] unknown directive "concat_mac_files" in /usr/local/nginx/conf/nginx.conf:35

自动生成的配置文件nginx.conf中
concat_mac_files 笔误应该是 concat_max_files

使用concat合并CSS后，CSS里面的背景图片地址必须写成绝对地址吗？

如题，我写相对路径，图片是404的。
但像minify合并CSS的话，它会自动将相对路径做出更改为正确的相对路径。
不知你们的concat有这功能不？还是只能写绝对路径。

nginx/1.7.6新版本对JS使用标准MIME-Type（application/javascript）导致的400 Bad Request

复现过程：删除nginx的目录：rm /usr/local/nginx/ -rf，重新安装Nginx的1.7.6版本，并添加本模块，按说明文档配置后，按照说明访问css文件没有问题，但访问js文件时候，会出现400的错误提示。

原因是：Nginx1.7.6使用了标准的MIME-Type：application/javascript，而本模块的代码中，写的是application/x-javascript的类型，不匹配，所以导致这个问题

解决方案：1、修改nginx的mime.types配置文件，修改js的文件类型为application/x-javascript；2、修改模块的源文件，重新编译nginx并安装，统一为application/javascript；3、等待这个模块更新

concat语法支持问题

我们之前一直在用tengine的concat模块,之前都是手动修改源文件里面的多个script标签到一个??格式的标签的
但是最近加入了一个自动编译的脚本,会遍历所有的script标签,按照LCP(longest common prefix)算法提取连接里面的前缀,再拼接到地址里面去,
比如有两个script标签

<script type="text/javascript" src="/Public/Vendor/jquery.func1.js"></script>
<script type="text/javascript" src="/Public/Vendor/jquery.func2.js"></script>

之前人工合并的时候会改写成:

<script type="text/javascript" src="/Public/Vendor/??jquery.func1.js,jquery.func2.js"></script>

执行就是好的,但是采用了自动脚本的LCP语法之后,会被自动替换成

<script type="text/javascript" src="/Public/Vendor/jquery.func??1.js,2.js"></script>

这样这个模块就不支持了.
我在文档里面没有看到说明说两个问号比武放在/这个位置的正后面?

支持下Proxy吧：）

这个，应该还算蛮实用吧。
不会写代码的我，求助叔度大师们。。。

配置项不能指定root目录吗？

我的静态文件分了好几个目录，想请求url为http://127.0.0.1/static/??js/a.js,js1/b.js,js2/c.js
然后配置location拦截/static/，但是我的目录js，js1，js2并不在static目录下面，所以想在location下面配置个root指定根目录，发现没有效果，请问现在是不允许或还不支持这么配置吗？

Browser specific gzip compression ?

Has nginx http concat module been tested to see if gzip compression across browsers is working for css and jss files ?

I tested a few browsers at http://nontroppo.org/tools/gziptest/?url=http%3A%2F%2Fghost.centminmod.com%2Fassets%2Fcss%2F%3F%3Fscreen.css%2Cnormalize.css&uas=&onlyheaders=on&showbody=on and it seems some browsers don't recognise gzip compression for concat urls like http://ghost.centminmod.com/assets/css/??screen.css,normalize.css and http://centminmod.com/css/??bootstrap.css,bootstrap-responsive.css,style.css,ticker.css both report as uncompressed in Opera 15 but not in Chrome 30 developer tools ?

Return delimiter when file was not found

Hi, we want to use the http-concat module to batch get multiple files (not only css/javascript) and we're currently using a custom concat_delimiter to be able to split on that and get the body for each file individually.

However, when requesting a file that doesn't exist (we're using concat_ignore_file_error on) the file gets ignored but no delimiter gets added. At that point our UI has no idea which file wasn't found.
Would it be feasable to add an empty body (even better: have something configurable like concat_not_found_body) and the delimiter? That way, if you request 4 files, you always get 3 delimiters. (and thus 4 separate bodies)

【已解决】一直遇到400错误，合并请求的时候

用这个东西合并请求，遇到400 错误。排查了很久，

最后是没有在配置文件里加 include mime.types;

类似下面这样

http {
include mime.types;
}

贴出来，希望可以帮到其他人

alibaba / nginx-http-concat Goto Github PK

nginx-http-concat's Introduction

HTTP Concatenation module for Nginx

Introduction

Configuration example

Module directives

Installation

Tagging releases

Other tengine modules on Github

Other builds

Acknowledgments

License

nginx-http-concat's People

Contributors

Stargazers

Watchers

Forkers

nginx-http-concat's Issues

Recommend Projects

Recommend Topics

Recommend Org