alibaba / nginx-tfs Goto Github PK

Hi, I noticed that there are APIs for creating and manipulating directories, but missing docs describing how to store a file under dir, So, could you give me some hints for this? thanks.

Hi,
我正在使用TFS，想使用restapi，发现没办法编译过去。
cc1: warnings being treated as errors
../nginx-tfs-master//ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_tackle_log’中:
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误：隐式声明函数‘ngx_log_create’
../nginx-tfs-master//ngx_http_tfs_module.c:517: 错误：赋值时将整数赋给指针，未作类型转换
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误：隐式声明函数‘ngx_log_set_levels’
../nginx-tfs-master//ngx_http_tfs_module.c:526: 错误：返回时将整数赋给指针，未作类型转换
make[1]: *** [objs/addon/nginx-tfs-master/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/root/tengine-2.1.0'
make: *** [build] 错误 2

去掉MakeFile 的-Werror 选项会出现下面的错误：
objs/addon/nginx-tfs-master/ngx_http_tfs_module.o: In function ngx_http_tfs_tackle_log': /root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:517: undefined reference to ngx_log_create'
/root/tengine-2.1.0/../nginx-tfs-master//ngx_http_tfs_module.c:526: undefined reference to `ngx_log_set_levels'

请问我该设置什么才能编译过去？

发现tfs_pass只能配置在 "/" location下面，如果该nginx不光处理tfs，还可能有其他业务的配置，感觉让tfs走”/“ location有点不妥。

类似的问题：

http://code.taobao.org/p/tfs/issue/1312/

相关信息：

系统信息

$ uname -a
Linux tfs004 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

ngx_openresty 版本

$ ./nginx -V
nginx version: ngx_openresty/1.2.7.1
built by gcc 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty-debug/nginx --with-debug --with-cc-opt=-O0 --add-module=../ngx_devel_kit-0.2.18 --add-module=../iconv-nginx-module-0.10rc7 --add-module=../echo-nginx-module-0.42 --add-module=../xss-nginx-module-0.03rc9 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.22rc8 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.02 --add-module=../ngx_postgres-1.0rc2 --add-module=../srcache-nginx-module-0.19 --add-module=../ngx_lua-0.7.16 --add-module=../headers-more-nginx-module-0.19 --add-module=../array-var-nginx-module-0.03rc1 --add-module=../memc-nginx-module-0.13rc3 --add-module=../redis2-nginx-module-0.09 --add-module=../redis-nginx-module-0.3.6 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.12rc10 --add-module=../rds-csv-nginx-module-0.05rc2 --with-ld-opt=-Wl,-rpath,/home/server/openresty-debug/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs

nginx-tfs 模块最新的 commit

$ git log -1
commit d1c8349
Merge: dcb7507 83d9a33
Author: Zheng Cen [email protected]
Date: Thu Jan 17 03:03:57 2013 -0800

Merge pull request #12 from monadbobo/fix_upstream

Fix upstream

重现问题的步骤

(1) 使用下面的 nginx.conf 启动 nginx

nginx.conf: https://gist.github.com/harveyzh/5055847#file-nginx-conf

(2) ngx.location.capture 指令直接调用 nginx-tfs 模块会一直 block

测试命令：
$ curl -i 127.0.0.1/test_block
^C

相关配置片段：

    location /test_block {
        content_by_lua '
            res = ngx.location.capture("/v1/tfs/T17yCTByJT1RCvBVdK")

            if res.status == ngx.HTTP_OK then
                ngx.print(res.body)
            else
                ngx.exit(res.status)
            end
        ';
    }

debug log: https://gist.github.com/harveyzh/5055847#file-error-log-block

block时的 debug log 停在

2013/02/28 18:41:06 [debug] 29836#0: *6 http copy filter: 0 "/v1/tfs/T17yCTByJT1RCvBVdK?"
2013/02/28 18:41:06 [debug] 29836#0: *6 http finalize request: 0, "/v1/tfs/T17yCTByJT1RCvBVdK?" a:0, c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua run post subrequest handler, rc:0 c:2
2013/02/28 18:41:06 [debug] 29836#0: *6 lua restoring write event handler
2013/02/28 18:41:06 [debug] 29836#0: *6 http wake parent request: "/test_block?"
2013/02/28 18:41:06 [debug] 29836#0: keepalive dummy handler
2013/02/28 18:41:06 [debug] 29836#0: timer delta: 1
2013/02/28 18:41:06 [debug] 29836#0: posted events 0000000000000000
2013/02/28 18:41:06 [debug] 29836#0: worker cycle
2013/02/28 18:41:06 [debug] 29836#0: epoll timer: -1

(3) ngx.location.capture 指令间接通过 proxy_pass 调用 nginx-tfs 模块则 ok

测试命令：
$ curl -i 127.0.0.1/test_ok
HTTP/1.1 200 OK
Server: ngx_openresty
Date: Thu, 28 Feb 2013 10:59:44 GMT
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Connection: keep-alive

test!

相关配置片段：

    location /tfs/ {
        proxy_pass http://127.0.0.1/v1/tfs/;
    }

    location /test_ok {
        content_by_lua '
            res = ngx.location.capture("/tfs/T17yCTByJT1RCvBVdK")

            if res.status == ngx.HTTP_OK then
                ngx.print(res.body)
            else
                ngx.exit(res.status)
            end
        ';
    }

debug log: https://gist.github.com/harveyzh/5055847#file-error-log-ok

如题。restful api 中有选择存储为大文件或者小文件，我们根据什么规则存储为大文件或小文件呢？这两种存储方式有什么区别呢？

日志：

10.81.254.51 - - [18/Aug/2014:17:04:47 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 909787 "-" "Golang rpc package" "-" 7.765 7.312
10.81.254.51 - - [18/Aug/2014:17:04:48 +0800] "GET /tfs/v1/tfs/T1Bvh8BTZ_1RCvBVdK HTTP/1.1" 200 863255 "-" "Golang rpc package" "-" 1.555 1.342

返回的大小相差不小。

nginx -V：

nginx version: ngx_openresty/1.4.3.9
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
TLS SNI support enabled
configure arguments: --prefix=/home/server/openresty/nginx --add-module=../ngx_devel_kit-0.2.19 --add-module=../iconv-nginx-module-0.10 --add-module=../echo-nginx-module-0.50 --add-module=../xss-nginx-module-0.04 --add-module=../ngx_coolkit-0.2rc1 --add-module=../set-misc-nginx-module-0.23 --add-module=../form-input-nginx-module-0.07 --add-module=../encrypted-session-nginx-module-0.03 --add-module=../ngx_postgres-1.0rc3 --add-module=../srcache-nginx-module-0.24 --add-module=../ngx_lua-0.9.3 --add-module=../headers-more-nginx-module-0.24 --add-module=../array-var-nginx-module-0.03 --add-module=../memc-nginx-module-0.14 --add-module=../redis2-nginx-module-0.10 --add-module=../redis-nginx-module-0.3.7 --add-module=../auth-request-nginx-module-0.2 --add-module=../rds-json-nginx-module-0.13 --add-module=../rds-csv-nginx-module-0.05 --with-ld-opt=-Wl,-rpath,/home/server/openresty/luajit/lib --with-http_ssl_module --with-http_stub_status_module --add-module=/home/makerpm/git/nginx-auth-ldap --add-module=/home/makerpm/git/nginx_upstream_hash --add-module=/home/makerpm/git/ngx_http_substitutions_filter_module --add-module=/home/makerpm/git/nginx-tfs --add-module=/home/makerpm/git/nginx-upstream-fair --with-pcre=/home/makerpm/tar/pcre-8.34 --with-pcre-jit --with-openssl=/home/makerpm/tar/openssl-1.0.1h

nginx-tfs 模块是从 tenginx 里面拷贝出来的。

请教：在通过http://xxx:7500/v1/tfs 上传文件时，如何加鉴权？
比如想在客户端http POST时在head中加一个从用户系统得到的token,然后请求TFS时如何鉴权这个token?
目前的想法独立做一个上传文件的WEB应用，在这里做head token鉴权,通过后再转发到TFS上。这样的话就多了一个应用。我们nginx-tfs目前有没有类似的鉴权方法？盼复~~

/usr/local/src/nginx-tfs/src/ngx_http_tfs.c: 在函数‘ngx_http_tfs_process_upstream_request’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs.c:874: 错误：提供给函数‘c->recv_chain’的实参太少
make[1]: *** [objs/addon/src/ngx_http_tfs.o] 错误 1
make[1]: *** 正在等待未完成的任务....
cc1: warnings being treated as errors
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c: 在函数‘ngx_http_tfs_log’中:
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误：隐式声明函数‘ngx_log_create’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:684: 错误：赋值时将整数赋给指针，未作类型转换
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误：隐式声明函数‘ngx_log_set_levels’
/usr/local/src/nginx-tfs/src/ngx_http_tfs_module.c:693: 错误：返回时将整数赋给指针，未作类型转换
make[1]: *** [objs/addon/src/ngx_http_tfs_module.o] 错误 1
make[1]: Leaving directory `/usr/local/src/nginx-1.10.3'
make: *** [build] 错误 2

编译一直报这个错，这里用的是nginx-1.10.3版本，换了其它版本以及tengine也不行

我找其中一个等待shutdown的进程，跟踪下结果如下，发现有Resource temporarily unavailable不知道什么原因，这个问题只有使用nginx-tfs时才会出现，版本tengine-2.1.0和tengine2.1.2上都会出现，有的进程shutdown状态持续到几个月。
strace -p 31072

Process 31072 attached - interrupt to quit
epoll_wait(27, {}, 512, 7029) = 0
epoll_wait(27, {}, 512, 1) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 2682
ioctl(2682, FIONBIO, [1]) = 0
epoll_ctl(27, EPOLL_CTL_ADD, 2682, {EPOLLIN|EPOLLOUT|EPOLLET|0x2000, {u32=495118000, u64=140699328767664}}) = 0
connect(2682, {sa_family=AF_INET, sin_port=htons(5100), sin_addr=inet_addr("172.16.0.198")}, 16) = -1 EINPROGRESS (Operation now in progress)
epoll_wait(27, {{EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9999) = 1
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0f\20\r\0\0\0\0\0gi-\3632\0\0\0002-31"..., 293}], 1) = 293
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9997) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0f\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed65f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
getsockopt(2682, SOL_SOCKET, SO_ERROR, [102868521188327424], [4]) = 0
writev(2682, [{"TFSN\r\1\0\0D\0\2\0g\20\r\0\0\0\0\0\23\310\36232\0\0\0001-31"..., 293}], 1) = 293
readv(2682, 0x7fff6b9ed4c0, 2) = -1 EAGAIN (Resource temporarily unavailable)
epoll_wait(27, {{EPOLLIN|EPOLLOUT, {u32=495118000, u64=140699328767664}}}, 512, 9996) = 1
readv(2682, [{"TFSN\1\0\0\0E\0\2\0g\20\r\0\0\0\0\0\253\306Hl", 24}, {"\0\311\377\377\27\0\0\0execute message failed\0\2"..., 20971520}], 2) = 25
recvfrom(2682, 0x7fff6b9ed60f, 1, 2, 0, 0) = -1 EAGAIN (Resource temporarily unavailable)
close(2682) = 0

tfs递归创建文件时怎么第一次操作是创建目录而不是文件？

想创建/cc/cc/cc文件，却创建/cc/cc/cc目录。
[root@tfsdatasvr1 ~]# curl -i -XPOST http://tfs.test.com/v2/tappkey00001/1/1030/file/cc/cc/cc?recursive=1
HTTP/1.1 201 Created
Server: Tengine/2.1.2
Date: Mon, 27 Feb 2017 03:06:59 GMT
Content-Length: 0
Connection: keep-alive

[root@tfsdatasvr1 ~]# curl http://tfs.test.com/v2/tappkey00001/1/1030/dir/cc
[
{
"NAME": "cc",
"PID": 43489476,
"ID": 43489477,
"SIZE": 0,
"IS_FILE": false,
"CREATE_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"MODIFY_TIME": "Mon, 27 Feb 2017 02:47:12 GMT",
"VER_NO": 0
}
]

执行创建文件(create_file) POST /v2/appkey/appid/uid/file/file_name HTTP/1.1接口可以正确返回201状态，但是在执行写文件(write_file) PUT /v2/appkey/appid/uid/file/file_name HTTP/1.1接口时，nginx err.log中出现如下信息:
2017/12/06 17:50:14 [error] 34442#0: *1257 error, cluster id conflict: 1(ns) <> 54(rcs) while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500" 2017/12/06 17:50:14 [error] 34442#0: *1257 http tfs process name server request failed while reading response header from tfs, client: 192.168.10.57, server: localhost, request: "PUT /v2/tappkey00001/1/21312/file/file1?offset=213213213&size=2436 HTTP/1.1", host: "192.168.10.128:7500"
请大神帮忙解惑！小弟不胜感激!

您好，请教下upstream动态更改 怎么实现的？

问题描述：
通过nginx-tfs上传大于2m的文件到TFS失败，nginx-tfs返回500。而使用TFS client可以正常写入大于2m的文件。在dataserver的日志发现， dataserver 解析nginx-tfs传来的数据报，取到数据包长度为负值。会议nginx-tfs存在bug。

nginx-tfs 相关的配置如下：
## TFS
tfs_send_timeout 3s;
tfs_read_timeout 3s;
tfs_connect_timeout 3s;
tfs_body_buffer_size 10M;
tfs_block_cache_zone size=256M;
tfs_keepalive max_cached=128 bucket_count=16;

nginx的debug日志如下：
2013/10/09 16:15:20 [info] 19460#0: 8 meta segment: block_id: 0, fileid: 0, seq_id: 0, suffix: 0, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 get block info from ns while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting name server, addr: 10.33.56.125:8108
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state name server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs finalize state data server, 0 while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs..com"
2013/10/09 16:15:20 [info] 19460#0: *8 http tfs process next peer is data server, addr: 10.33.56.198:8010 while reading response header from tfs, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**.com"
2013/10/09 16:15:20 [info] 19460#0: 8 write segment index 0, block id: 2404301, file id: 2354, offset: 0, length: 2097152, crc: 2021805896 while connecting server, client: 127.0.0.1, server: tfs.**.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.__.com"
2013/10/09 16:15:20 [debug] 19460#0: *8 connecting data server, addr: 10.33.56.198:8010
2013/10/09 16:15:20 [error] 19460#0: *8 readv() failed (104: Connection reset by peer) while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs.**__.com"
2013/10/09 16:15:20 [error] 19460#0: *8 recv chain error while reading response header from tfs, client: 127.0.0.1, server: tfs._.com, request: "POST /v1/tfs HTTP/1.1", host: "tfs._.com"

TFS 使用2.2.13 版本。dataserver的报错日志为：
2013-10-09 18:07:22] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: 69fc7285<>4d534654,4e534654, dataLen: 432408354
[2013-10-09 18:07:22] ERROR reply (base_packet.cpp:221) [139896073172736] post packet failure, server: 10.33.56.23:55566, pcode:1
[2013-10-09 18:07:22] ERROR callback (dataservice.cpp:854) [139896073172736] write data fail. filenumber: 14152088283100828036, blockid: 2399407, fileid: 2915, version: 2914, leaseid: 1198248338, role: master

[2013-10-09 17:28:43] ERROR getPacketInfo (base_packet_streamer.cpp:86) [139896236799744] stream error: ebe940fc<>4d534654,4e534654, dataLen: -1052153523
[2013-10-09 17:28:43] ERROR reply (base_packet.cpp:221) [139896167581440] post packet failure, server: 10.81.102.121:54715, pcode:1
[2013-10-09 17:28:43] ERROR callback (dataservice.cpp:854) [139896167581440] write data fail. filenumber: 14152088283100823092, blockid: 2407850, fileid: 1621, version: 1620, leaseid: 1197348515, role: master

PUT /v2/tfs/1/1234/file/file_1 HTTP/1.1 写文件报500错误，其他所有测试都过了，不知道什么原因 ，目录创建删除，文件创建删除都没有问题，就是写文件时报错500，不知道什么原因。

1.在post数据的时候如果不加suffix=这个参数会提示400错误，然后加了之后就可以提交数据了，然后我多加了个simple_name参数就又报400错误，然后看你api说明的时候这两个不加的时候也应该是可以的，加了两个也可以，现在这个算是bug吗？
2.我用post提交完数据之后返回文件名了，然后用这个文件名想用restful api的get接口搞下来，但始终无效，报404 not found，然后用后台的nameserver查看那个文件名发现是已经提交上去了的，请问这是什么原因？

objs/addon/src/ngx_http_tfs_module.o: In function `ngx_http_tfs_log':
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:686: undefined reference to `ngx_log_create'
/home/makerpm/git/nginx-tfs/src/ngx_http_tfs_module.c:695: undefined reference to `ngx_log_set_levels'
collect2: ld returned 1 exit status
gmake[2]: *** [objs/nginx] Error 1
gmake[2]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake[1]: *** [build] Error 2
gmake[1]: Leaving directory `/home/makerpm/tar/ngx_openresty-1.7.0.1/build/nginx-1.7.0'
gmake: *** [all] Error 2

ngx_log_create 被不存在了。