replay / ngx_http_consistent_hash Goto Github PK

a module which enables the nginx to use the same consistent hashing distribution for memcache servers as the php memcache module

C 100.00%

ngx_http_consistent_hash's People

Contributors

Stargazers

Watchers

ngx_http_consistent_hash's Issues

Question: How does consistent hash interact with backup upstreams?

How does consistent hash interact with backup upstreams?

If the target server for a hashed path is down, does nginx use a backup server/another server in the upstream block? Or, does it just respond with a 5XX error? If it proxies to another server, is the new server now marked for consistent hashing of the path?

Thank you!

how to realize it with golang

Why are we reducing hash lookup to MMC_CONSISTENT_BUCKETS ?

    for (i = 0; i < us->servers->nelts; i++) {
        for (j = 0; j < server[i].naddrs; j++) {
            for (k = 0; k < (MMC_CONSISTENT_POINTS * server[i].weight); k++) {
                ngx_snprintf(hash_data, HASH_DATA_LENGTH, "%V-%ui%Z", &server[i].addrs[j].name, k);
                continuum->nodes[continuum->nnodes].sockaddr = server[i].addrs[j].sockaddr;
                continuum->nodes[continuum->nnodes].socklen = server[i].addrs[j].socklen;
                continuum->nodes[continuum->nnodes].name = server[i].addrs[j].name;
                continuum->nodes[continuum->nnodes].name.data[server[i].addrs[j].name.len] = 0;
                continuum->nodes[continuum->nnodes].point = ngx_crc32_long(hash_data, ngx_strlen(hash_data));
                continuum->nnodes++;
            }
        }
    }

    qsort(continuum->nodes, continuum->nnodes,
            sizeof(ngx_http_upstream_consistent_hash_node),
            (const void*) ngx_http_upstream_consistent_hash_compare_continuum_nodes);

    for (i = 0; i < MMC_CONSISTENT_BUCKETS; i++) {
        buckets->buckets[i] =
            ngx_http_upstream_consistent_hash_find(continuum, step * i);
    }

What is the advantage of calculating MMC_CONSISTENT_POINTS for each and then reducing all to MMC_CONSISTENT_BUCKETS ?.
I mean lets say there are 4 address we are doing 4 * MMC_CONSISTENT_POINTS and reducing to MMC_CONSISTENT_BUCKETS and doing a lookup only on the buckets { How are we sure the server is spready equally in this bucket ? }, what advantage are we getting with the same ?.

rgds
Nandhan

openresty[16290]: segfault at 0 ip (null) sp 00007fff403faf98 error 14 in nginx[400000+1da000]

Centos Version:

[root@tvvmjt0128 ~]# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

OpenResty Infos:

[egwnode@tvvmjt0128 coredump]$ openresty -V
nginx version: openresty/1.19.9.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/usr/local/openresty/nginx --with-debug --with-cc-opt='-DNGX_LUA_USE_ASSERT -DNGX_LUA_ABORT_AT_PANIC -O2' --add-module=../ngx_devel_kit-0.3.1 --add-module=../echo-nginx-module-0.62 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.32 --add-module=../ngx_lua-0.10.20 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.09 --add-module=../ngx_stream_lua-0.0.10 --with-ld-opt=-Wl,-rpath,/usr/local/openresty/luajit/lib --with-pcre-jit --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-stream --with-stream_ssl_preread_module --with-http_ssl_module --with-http_gzip_static_module --add-module=/data/software/openresty-1.19.9.1/ngx_http_consistent_hash-master --with-stream

Nginx Config:

upstream vpn-backends {
    consistent_hash $proxy_add_x_forwarded_for;
    server 10.x.x.2:8081;
    server 10.x.x.1:8081;
}

server {
    listen 80;
    location /vpn/ {
        proxy_http_version 1.1;
        proxy_set_header  Connection      "";
        proxy_set_header  X-Real-IP       $remote_addr;
        proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header  Host            $http_host;
        proxy_cookie_path   /buap /vpn;
        proxy_pass https://vpn-backends/;
  }
}

When I complete the relevant configuration to access, Nginx has the following error：

2022/03/30 00:11:36 [alert] 16288#16288: worker process 16290 exited on signal 11 (core dumped)

When I use dmesg | grep nginx, I get the following error:

[29426336.444125] openresty[9722]: segfault at 0 ip           (null) sp 00007fff5ba57498 error 14 in nginx[400000+1da000]
[29426363.556545] openresty[9723]: segfault at 0 ip           (null) sp 00007fff5ba57498 error 14 in nginx[400000+1da000]
[29426423.957289] openresty[9829]: segfault at 0 ip           (null) sp 00007fff5ba574c8 error 14 in nginx[400000+1da000]
[29426882.473798] openresty[12564]: segfault at 0 ip           (null) sp 00007ffde606af78 error 14 in nginx[400000+1da000]
[29426884.310026] openresty[12563]: segfault at 0 ip           (null) sp 00007ffde606af78 error 14 in nginx[400000+1da000]
[29426893.608951] openresty[12666]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29426894.408055] openresty[12679]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29426902.102552] openresty[12716]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29426908.099369] openresty[12719]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29426908.422015] openresty[12751]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29426908.827020] openresty[12776]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29427293.224662] openresty[12781]: segfault at 0 ip           (null) sp 00007ffde606afa8 error 14 in nginx[400000+1da000]
[29427632.713972] openresty[16290]: segfault at 0 ip           (null) sp 00007fff403faf98 error 14 in nginx[400000+1da000]

I don't know why this is? Do you have any good solution? thanks.

how to hash with args in $request_body

consistent_hash $arg_*;
this can hash with args in url but not in $request_body

Building against Nginx 1.0.6 fails

I just tried to build the module against Nginx 1.0.6 on Ubuntu 10.04 and am getting:

cc1: warnings being treated as errors
/src/ngx_http_consistent_hash/ngx_http_upstream_consistent_hash_module.c: In function ‘ngx_http_upstream_init_consistent_hash’:
/src/ngx_http_consistent_hash/ngx_http_upstream_consistent_hash_module.c:128: error: assignment makes integer from pointer without a cast
make[1]: *** [objs/addon/ngx_http_consistent_hash/ngx_http_upstream_consistent_hash_module.o] Error 1
make[1]: Leaving directory `/src/nginx-1.0.6'
make: *** [build] Error 2

how to hash with the header？

I added a custom field to the request header.
the file is : session
my nginx config:

location /abc { proxy_pass http://d/logServer/; }

`
upstream d {

    hash $http_session consistent;
   server ip:端口;
   server ip:端口;

}
`

Matching up with Python

I'm trying to use this module with python and following this blog post to make python consistent hashing behave like php's (http://www.ruturaj.net/python-php-memcache-hash) I am still unable to get a match. I'm actually using redis and not memcache and implemented a system much like that found in python-memcache, here is my code which I plan on putting up on github once its in working order. http://pastie.org/1292936

how to apply this hash method to tcp load balance

I notice that this method is used for http load balancing, but I decide to apply it to tcp load balance. Could you give me some guidance? Thanks a lot!

replay / ngx_http_consistent_hash Goto Github PK

ngx_http_consistent_hash's People

Contributors

Stargazers

Watchers

Forkers

ngx_http_consistent_hash's Issues

Centos Version:

OpenResty Infos:

Nginx Config:

Recommend Projects

Recommend Topics

Recommend Org