xplicit / hyperfastcgi Goto Github PK

View Code? Open in Web Editor NEW

131.0 19.0 49.0 351 KB

Performant nginx to mono fastcgi server

License: MIT License

Shell 3.88% C# 62.60% C 24.16% Makefile 3.21% M4 3.99% Smarty 1.28% Dockerfile 0.88%

mono-server-fastcgi nginx web-server mono asp

hyperfastcgi's People

Contributors

Stargazers

Watchers

hyperfastcgi's Issues

GetPathInfo () always return String.Empty

Hi,

GetPathInfo () always return String.Empty
Is there a good reason for it?
https://github.com/xplicit/HyperFastCgi/blame/master/src/Mono.WebServer.HyperFastCgi/WorkerRequest.cs#L189

При обновлении сервиса "на горячую" HyperFastCgi либо падает, либо зависает и не выдаёт результат

Возможно есть правильная процедура обновления bin-файлов?

HyperFastCgi Performace Degradation

Hello,
I've been evaluating HyperFastCgi in the hope it will improve performance compared to the stock Mono implementation. Your excellent blog series (http://forcedtoadmin.blogspot.com/2013/11/servicestack-performance-in-mono-p1.html) encouraged me to try HFC.

Unfortunately, I've encountered severe performance degradation after a short while during testing with ApacheBench.

The following test URL is handled by our webapp, which simply responds by echoing the string from the last path segment (i.e. "test").

ab -kc 20 -t 360 http://internal-ec2-instance.mycompany.com/helloworld/api/hello/echo/test

The table below shows the degradation over several test runs:

Test Run	Requests	Time (sec)	Requests/Second	CPU %
1	50000	102.760	486.57	~165%
2	50000	94.819	527.32	~165%
3	50000	133.470	374.62	~135%
4	50000	180.357	277.23	~40-80%
5	49988	360.012	138.85	~0-50%
6	2963	360.022	8.23	~0-8%

The helloworld test service was started fresh before test 1, and not restarted between tests.

Note that as performance degraded, CPU utilization by the HyperFastCgi process decreased, so it doesn’t appear to be CPU bound.

Also, periodically while running these tests, I checked open connections owned by mono using:

sudo netstat -ap | grep mono | wc –l

The highest value I got during a test was 84, and usually was closer to 40-50. I actually think that seems low, but at least it seems to show that connections aren’t being left open unnecessarily.

The test above was run with mono –debug; however, other tests were run without this flag, with very similar performance results. Without –debug, it took a little while longer for the degradation to show up, and the lowest measured throughput was ~26 requests/sec (compared to ~8 with –debug). CPU utilization was the same, but there was slightly lower memory utilization by the HFC process.

I’ve observed similar behavior with various HFC and Nginx configurations, but for the tests above I used the following configs.

HFC (helloworld.config):

<configuration>
        <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
                <!-- Host factory defines how host will be created. SystemWebHostFactory creates host in AppDomain in standard ASP.NET way -->
                <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
                <!-- <threads> creates threads at startup. Value "0" means default value -->
                <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
                <!--- Sets the application host root directory -->
                <!-- <root-dir>/path/to/your/dir</root-dir> -->
        </server>
        <listener type="HyperFastCgi.Listeners.NativeListener">
                <apphost-transport type="HyperFastCgi.Transports.NativeTransport" />
                <protocol>Unix</protocol>
                <address>/tmp/helloworld</address>
        </listener>
    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
                <log level="Debug" write-to-console="true" />
                <add-trailing-slash>false</add-trailing-slash>
    </apphost>
    <web-applications>
        <web-application>
                <name>HelloWorld.Web</name>
                <vhost>internal-ec2-instance.mycompany.com</vhost>
                <vport>80</vport>
                <vpath>/helloworld</vpath>
                <path>/opt/mycompany/HelloWorld/HelloWorld.Web</path>
        </web-application>
    </web-applications>
</configuration>

Nginx Configs:

/etc/nginx/nginx.conf

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;

events {
        worker_connections 768;
        multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;    
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Included via /etc/nginx/sites-enabled/*;

upstream fastcgi_backend {
  server unix:/tmp/helloworld;
  keepalive 32;
}

server {
  listen 80 default_server;
  listen [::]:80 default_server ipv6only=on;

  include /etc/nginx/locations/*;
  include /opt/mycompany/nginx/fastcgi_common_params;
}

Included via /etc/nginx/locations/*;

  location /helloworld {
    fastcgi_pass fastcgi_backend;
    fastcgi_keep_conn on;
  }

Included via /opt/mycompany/nginx/fastcgi_common_params;

fastcgi_param  QUERY_STRING        $query_string;
fastcgi_param  REQUEST_METHOD      $request_method;
fastcgi_param  CONTENT_TYPE        $content_type;
fastcgi_param  CONTENT_LENGTH      $content_length;
fastcgi_param  SCRIPT_NAME         $fastcgi_script_name;
fastcgi_param  REQUEST_URI         $request_uri;
fastcgi_param  DOCUMENT_URI        $document_uri;
fastcgi_param  DOCUMENT_ROOT       $document_root;
fastcgi_param  SERVER_PROTOCOL     $server_protocol;
fastcgi_param  GATEWAY_INTERFACE   CGI/1.1;
fastcgi_param  SERVER_SOFTWARE     nginx/$nginx_version;
fastcgi_param  REMOTE_ADDR         $remote_addr;
fastcgi_param  REMOTE_PORT         $remote_port;
fastcgi_param  SERVER_ADDR         $server_addr;
fastcgi_param  SERVER_PORT         $server_port;
fastcgi_param  SERVER_NAME         $server_name;
fastcgi_param  HTTPS               $https if_not_empty;
fastcgi_param  PATH_INFO           "";
fastcgi_param  SCRIPT_FILENAME     $document_root$fastcgi_script_name;

We’re executing HFC as a service via init.d script. In the script, we call HFC as follows (helloworld.config is the HFC config file above):

    sudo -u serviceuser -E hyperfastcgi4
    /config=/opt/mycompany/nginx/hyperfastcgi/helloworld.config
    /keepalive=true
    /logfile=/var/log/mycompany/hyperfastcgi_helloworld.log
    /loglevels=All
    /printlog=True

My test system is an EC2 m3.large instance running Ubuntu 12.4. The ApacheBench tests were executed on another, separate EC2 instance.

Our Mono version:

Mono JIT compiler version 3.12.1 (tarball Fri Mar  6 19:12:47 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
    TLS:           __thread
    SIGSEGV:       altstack
    Notifications: epoll
    Architecture:  amd64
    Disabled:      none
    Misc:          softdebug
    LLVM:          supported, not enabled.
    GC:            sgen

I built HyperFastCgi 0.4 from master, compiled with autogen.sh --prefix=/usr, then make and make install.

As a baseline, I ran ApacheBench at 500 concurrency to fetch a static page served by Nginx from the same host:

ab -c 500 -t 360 http://internal-ec2-instance.mycompany.com/helloworld-static/index.html

The above test achieved over 12000 requests/second. I’m sure it doesn’t fully eliminate Nginx as a possible root cause, but at least we can see that the host in general is capable of much higher throughput.

Other configuration variations I have tried, with similar results, include:
• Disable keepalive in nginx.conf
• Set keepalive to 60 in nginx.conf
• Unset multi_accept in nginx.conf
• Use ManagedFastCgiListener with CombinedFastCgiListenerTransport and CombinedAppHostTransport in HFC config.

I have tried relevant suggestions already posted here, including this: #23

I’ll confess that I’m new to Nginx+FastCGI configuration, and fairly new to Mono.

Unfortunately, I’m basically stuck. Any suggestions are welcome.

Thanks in advance, and kind regards, Matt

Periodic HTTP request delays, stoppage

This is an excellent piece of engineering. Thank you for all the work. I've been doing some benchmarking and running into weird delays serving a simple hello.aspx. Running on EC2 c3.xlarge, nginx/HFC. Load testing with ab -n 10000 -c 4 http://10.2.1.62/hello.aspx...

If I use your test host HyperFastCgi.AppHosts.Raw.RawHost, it is rock solid. I can server 7000 hits per second endlessly. No memory leak, no tcp glitches or delays.

But if I switch to HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost to serve a simple "<%= DateTime.Now; %> aspx, I start to get delays and timeouts in the requests. I see the following in HFC output...

libev.c:331: cmd_error(): Remote host on fd 15 timed out.
libev.c:328: cmd_error(): Remote host disconnected from fd 23.
....

I'm using HyperFastCgi.Listeners.NativeListener

This happens whether or not nginx fastcgi keepalive is enabled.

Any idea why serving simple aspx this way is having timeouts between nginx and HFC ?

When the timeouts start happening, curl -v shows the TCP connection to nginx is successful, and the HTTP GET command/headers are send, but there is no reply

Command line option /stopable is incorrectly spelled

It should be /stoppable.

HyperFastCGI threading/IO configuration vs. ASP.NET configuration.

Hi,

we're having an ASP.NET web service with ServiceStack which itself calls multiple other HTTP web services at self - around 3 HTTP calls to an external service per request to our service.

We're suffering from very bad performance there. The called external webservice should be able to handle 100k requests per second.
We cannot send more than around 5 per second, altough we're having 1000 requests per second to this service as well.
So it seems we're encountering deadlocks and full queues.

I found this: https://support.microsoft.com/en-us/kb/821268
It seems like this is exactly the symptoms we're having.

The question now is:
The HFC configuration let's us define all of these parameters: <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />.

Is this related to the configuration recommendation Microsoft is making on their page?

Or is this totally distinct and am I supposed to add maxWorkerThreads, maxIoThreads, minFreeThreads, minLocalRequestFreeThreads etc to our Web.config as well, because otherwise the (low) default values would be in affect for our web application?

Thanks!

Change in file detected and application stops responding

If a file is changed I get a message like

Change: Web.config

All future requests to the unix socket result in 504 gateway timeout (using nginx).

I suspect the application is supposed to reload however I couldn't find any relevant logic in hyperfastcgi. Where is the above message coming from?

Large responses with native transport again

Alt Linux 7.0.5 - Nginx 1.7.11 - Mono 3.12.1 - last HyperFastCGI from sources
Configuration is from issue #24

It seems that problem with lange responses still exists.

Steps for reproduce:

Run the large test.aspx (~60kb)
https://drive.google.com/file/d/0B3W5PyVh7-kXMzF3SHlSZ2JTMkE/view?usp=sharing
Refresh page in your browser 10-20x times

Sometimes error will occur:
libev.c:322: cmd_error(): A socket error (0x21) occurred on fd 14.
libev.c:180: shutdown_cmdsocket(): Error shutting down client connection on fd 14: 107 (Конечная точка передачи не подсоединена)

Nginx log:
2015/04/23 18:42:08 [crit] 14492#0: *2820 open() "/usr/local/nginx/fastcgi_temp/1/27/0000000271" failed (13: Permission denied) while reading upstream, client: 192.168.68.11, server: foxtrot, request: "GET /test.aspx HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "..."

Installation problem (HyperFastCgi from source on FreeBSD 10.2)

git clone https://github.com/xplicit/HyperFastCgi.git

Установил gcc (во FreeBSD системный компилятор Clang):

cd /usr/ports/lang/gcc5/
make config install clean

Сделал алиас gcc:

cd /usr/local/bin/gcc
gcc-ar5*     gcc-nm5*     gcc-ranlib5* gcc5*        
ln -s gcc5 gcc

Запустил скрипт:

cd ./HyperFastCgi/
./autogen.sh --prefix=/usr
Running libtoolize ...
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'build/m4'.
libtoolize: copying file 'build/m4/libtool.m4'
libtoolize: copying file 'build/m4/ltoptions.m4'
libtoolize: copying file 'build/m4/ltsugar.m4'
libtoolize: copying file 'build/m4/ltversion.m4'
libtoolize: copying file 'build/m4/lt~obsolete.m4'
libtoolize: Consider adding '-I build/m4' to ACLOCAL_AMFLAGS in Makefile.am.
Running aclocal -I .  -I build/m4/shamrock -I build/m4/shave ...
Running automake --gnu  ...
configure.ac:25: installing './compile'
configure.ac:7: installing './missing'
src/libnative/Makefile.am: installing './depcomp'
Running autoconf ...
Running ./configure --prefix=/usr ...
checking build system type... amd64-unknown-freebsd10.2
checking host system type... amd64-unknown-freebsd10.2
checking target system type... amd64-unknown-freebsd10.2
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... nawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether make supports nested variables... (cached) yes
checking for pkg-config... /usr/local/bin/pkg-config
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/local/bin/ld
checking if the linker (/usr/local/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert amd64-unknown-freebsd10.2 file names to amd64-unknown-freebsd10.2     format... func_convert_file_noop
checking how to convert amd64-unknown-freebsd10.2 file names to toolchain format... func_convert_file_noop
checking for /usr/local/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/local/bin/ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... freebsd10.2 ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking pkg-config is at least version 0.9.0... yes
checking for GLIB... yes
checking for LIBEVENT... yes
checking for LIBEVENT_PTHREADS... yes
checking for MONO_MODULE... yes
checking for gmcs... no
checking for dmcs... /usr/local/bin/dmcs
checking for mono... /usr/local/bin/mono
checking for gacutil... /usr/local/bin/gacutil
checking for sn... /usr/local/bin/sn
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating build/Makefile
config.status: creating build/m4/Makefile
config.status: creating build/m4/shave/shave
config.status: creating build/m4/shave/shave-libtool
config.status: creating src/Makefile
config.status: creating src/HyperFastCgi/Makefile
config.status: creating src/HyperFastCgi/scripts/Makefile
config.status: creating src/libnative/Makefile
config.status: creating samples/Makefile
config.status: creating src/HyperFastCgi/scripts/hyperfastcgi4
config.status: executing depfiles commands
config.status: executing libtool commands

hyperfastcgi-0.4

  Build Environment
    Install prefix:          /usr
    Datadir:                 /usr/share
    Libdir:                  /usr/lib
    Build documentation:     
    Mono 2.0 compiler:       not found
    Mono 4.0 compiler:       /usr/local/bin/dmcs
    Target frameworks:       .NET 4.0

Тут начинаются проблемы:

make
Making all in src
Making all in HyperFastCgi
make[2]: "/root/HyperFastCgi/src/HyperFastCgi/Makefile" line 877: Need an operator
make[2]: "/root/HyperFastCgi/src/HyperFastCgi/Makefile" line 882: Need an operator
make[2]: "/root/HyperFastCgi/src/HyperFastCgi/Makefile" line 884: Need an operator
make[2]: Fatal errors encountered -- cannot continue
make[2]: stopped in /root/HyperFastCgi/src/HyperFastCgi
*** Error code 1

Stop.
make[1]: stopped in /root/HyperFastCgi/src
*** Error code 1

Stop.
make: stopped in /root/HyperFastCgi

exception on every request if duplicate param in fastcgi_params

If you have a duplicate parameter in /etc/nginx/fastcgi_params, then HFC outputs an exception with every web request inserting duplicate header into dictionary

Unix sockets Permissions

Hello, @xplicit , cool project!
Is it possible to specify the permissions for unix socket?
Like this:
... <address>//775@/tmp/onlyoffice.socket</address> ...

In NativeTransport.EndRequest: __pthread_mutex_lock: Assertion `mutex->data.owner == 0' failed.

Hi,

I'm still doing some excessive load testing to find out why we only reach low rps with our server implementation using HFC behind nginx 1.9.10.
After using Siege in benchmark mode HFC crashed with the following stracktrace.
I was using a simple stress test endpoint with ServiceStack which does simple web requests to another server of mine.

I hope this is helpful for you, I assume it could be a possible bug because happening on higher load:

...
libev.c:322: cmd_error():       A socket error (0x21) occurred on fd 47.
libev.c:180: shutdown_cmdsocket():      Error shutting down client connection on fd 47: 107 (Transport endpoint is not connected)
...
libev.c:317: cmd_error():       Remote host disconnected from fd 14.
libev.c:317: cmd_error():       Remote host disconnected from fd 46.
...
mono: ../nptl/pthread_mutex_lock.c:116: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed.
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) HyperFastCgi.Transports.NativeTransport.EndRequest (HyperFastCgi.Transports.NativeTransport,ulong,int,int) <0xffffffff>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.CloseConnection () <0x00058>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.FlushResponse (bool) <0x00024>
  at System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <0x000d0>
  at (wrapper remoting-invoke-with-check) System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <0xffffffff>
  at System.Web.HttpResponse.Flush (bool) <0x0021e>
  at System.Web.HttpApplication.OutputPage () <0x00085>
  at System.Web.HttpApplication.PipelineDone () <0x0011b>
  at System.Web.HttpApplication/<Pipeline>c__Iterator1.MoveNext () <0x06636>
  at System.Web.HttpApplication.Tick () <0x00057>
  at System.Web.HttpApplication.Start (object) <0x00272>
  at System.Web.HttpApplication.System.Web.IHttpHandler.ProcessRequest (System.Web.HttpContext) <0x00073>
  at System.Web.HttpRuntime.Process (System.Web.HttpWorkerRequest) <0x002e9>
  at System.Web.HttpRuntime.RealProcessRequest (object) <0x0008f>
  at System.Web.HttpRuntime.ProcessRequest (System.Web.HttpWorkerRequest) <0x0005d>
  at HyperFastCgi.AppHosts.AspNet.MonoWorkerRequest.ProcessRequest () <0x0009f>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.Process (HyperFastCgi.Interfaces.IWebResponse) <0x00014>
  at HyperFastCgi.Transports.BaseAppHostTransport/<AddBodyPart>c__AnonStorey0.<>m__0 () <0x000bb>
  at System.Threading.Tasks.Task.InnerInvoke () <0x0004a>
  at System.Threading.Tasks.Task.Execute () <0x00068>
  at System.Threading.Tasks.Task.ExecutionContextCallback (object) <0x00054>
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x0021a>
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x00028>
  at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task&) <0x00174>
  at System.Threading.Tasks.Task.ExecuteEntry (bool) <0x000da>
  at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0x00016>
  at System.Threading.ThreadPoolWorkQueue.Dispatch () <0x0021a>
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () <0x00010>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_bool (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

        /usr/bin/mono() [0x49d56c]
        /lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0) [0x7f6b7ac108d0]
        /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37) [0x7f6b7a88b067]
        /lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7f6b7a88c448]
        /lib/x86_64-linux-gnu/libc.so.6(+0x2e266) [0x7f6b7a884266]
        /lib/x86_64-linux-gnu/libc.so.6(+0x2e312) [0x7f6b7a884312]
        /lib/x86_64-linux-gnu/libpthread.so.0(pthread_mutex_lock+0x11b) [0x7f6b7ac0b38b]
        /usr/lib/libhfc-native.so.1(+0x55f7) [0x7f6b766755f7]
        /usr/lib/libhfc-native.so.1(end_request+0x112) [0x7f6b766758e2]
        [0x41d6467c]

Debug info from gdb:


=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

Aborted (core dumped)

Using latest HFC from master branch,

mono --version
Mono JIT compiler version 4.2.2 (Stable 4.2.2.30/996df3c Mon Feb 15 17:30:30 UTC 2016)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

cat /hyperfastcgi_1.conf
<configuration>
        <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">

                <!-- Host factory defines how host will be created. SystemWebHostFactory creates host in AppDomain in standard ASP.NET way -->
                <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>

                <!-- <threads> creates threads at startup. Value "0" means default value -->
                <threads min-worker="512" max-worker="4096" min-io="4" max-io="4096" />

        </server>

        <listener type="HyperFastCgi.Listeners.NativeListener">
                <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
                        <multithreading>Task</multithreading>
                </apphost-transport>
            <protocol>InterNetwork</protocol>
            <address>127.0.0.1</address>
            <port>9001</port>
        </listener>

    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
                <log level="Debug" write-to-console="true" />
                <add-trailing-slash>false</add-trailing-slash>
    </apphost>

</configuration>

app cannot be found - nginx on mac

While it could certainly be my configuration, i am matching samples but just can't seem to get a response. Nginx serves the static web page OK when no fastcgi is configured.

OS: Mac OS X Yosemite
NGinx: 1.9.4
Mono: 4.0.2
HyperFastCgi: 4 - built from master sources

server.config:

 <configuration>  
      <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">   
           <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>  
           <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />  
      </server>  
      <listener type="HyperFastCgi.Listeners.NativeListener">  
           <apphost-transport type="HyperFastCgi.Transports.NativeTransport"/>  
        <protocol>InterNetwork</protocol>  
        <address>127.0.0.1</address>  
        <port>9000</port>  
      </listener>  
   <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">  
           <log level="Debug" write-to-console="true" />  
           <add-trailing-slash>false</add-trailing-slash>  
   </apphost>  
   <web-applications>  
        <web-application>  
             <name>Web</name>  
             <vhost>localhost</vhost>  
             <vport>80</vport>  
             <vpath>/</vpath>  
             <path>/usr/local/var/www/</path>  
        </web-application>  
   </web-applications>  
 </configuration>

nginx.conf:

http {
upstream fastcgi_backend {
server 127.0.0.1:9000;
keepalive 32;
}
server {
listen 80 default_server;
server_name localhost;
access_log /var/log/web.log;
error_log /var/log/nginxerror.log;

location / {
root /usr/local/var/www/;
index index.html index.htm default.aspx Default.aspx;
fastcgi_index index.html;
fastcgi_keep_conn on;
fastcgi_pass fastcgi_backend;
include /usr/local/nginx/conf/fastcgi_params;
}
}
}
events { worker_connections 1024; }

run command:

hyperfastcgi4 /config=/usr/local/server.config /logfile=/usr/local/HyperFastCgi/logging.log /loglevels=Debug

Output:
[2015-09-17 22:18:02Z] Debug HyperFastCgi
libev.c:461: Listen(): libevent version: 2.0.22-stable
libev.c:475: Listen(): libevent is using kqueue for events.
fcgi-transport.c:394: parse_params(): Can't find app! HOST='localhost' port=80 path='/index.html'

Process crashes when duplicate headers are received in a request

I'm seeing the process crash due to duplicate key exception on some requests, and I was originally thinking maybe duplicate headers or url parameters, but running TCP dump doesn't show anything of concern.

This same code base works fine on fastcgi-mono-server4.

Unhandled Exception:
System.ArgumentException: An item with the same key has already been added.
at System.ThrowHelper.ThrowArgumentException (ExceptionResource resource) <0x41d7df30 + 0x0002b> in :0
at System.Collections.Generic.Dictionary2[TKey,TValue].Insert (System.Collections.Generic.TKey key, System.Collections.Generic.TValue value, Boolean add) <0x40ead100 + 0x0018b> in <filename unknown>:0 at System.Collections.Generic.Dictionary2[TKey,TValue].Add (System.Collections.Generic.TKey key, System.Collections.Generic.TValue value) <0x40ead0c0 + 0x00023> in :0
at Mono.WebServer.HyperFastCgi.Request.ParseParameters (System.Byte[] data, Boolean parseHeaders) <0x40fa70c0 + 0x001e7> in :0
at Mono.WebServer.HyperFastCgi.Request.AddParameterData (System.Byte[] data, Boolean parseHeaders) <0x40fa6e20 + 0x0009b> in :0
at Mono.WebServer.HyperFastCgi.NetworkConnector.ProcessRecord (Record record) <0x40fa62f0 + 0x00153> in :0
at Mono.WebServer.HyperFastCgi.NetworkConnector.ReceiveCallback (IAsyncResult ar) <0x40fa58e0 + 0x0058f> in :0

Anything hints at where the duplicate might be coming from? I'm not sure what it means when it says "ParseParameters". Parameters for what?

I'm running on the current 0.3.4 stable, and i'm not able to find any method matching ParseParameters (System.Byte[] data, Boolean parseHeaders) to investigate what might be happening in the source. :/

To build on Debian Jessie (8) you need libtool-bin

To build/install on Debian Jessie you need to apt-get install libtool-bin

Unable to call C# WebMethods

Ubuntu 14.04 LTS - Nginx 1.7.9 - Mono 3.12 - HyperFastCGI 0.4.4

Test function in jQuery:

function Test() {
            $.ajax({ 
                url: "Default.aspx/ServerCall",
                type: 'POST', 
                contentType: "application/json; charset=utf-8",
                dataType: "json",
                async: true,
                success: function (data, textStatus, jQxhr) 
                { alert(JSON.stringify(data)); }, 
                error: function (jqXhr, textStatus, errorThrown) 
                { alert(errorThrown); } })
        };

C#
[WebMethod]
public static string ServerCall()
{
      return "Hello World";
}

I always a a 500 response with the following error:

Error Message: System.Web.HttpException: Method 'POST' is not allowed when accessing file 'Default.aspx/ServerCall'
  at System.Web.DefaultHttpHandler.BeginProcessRequest (System.Web.HttpContext context, System.AsyncCallback callback, System.Object state) [0x00000] in <filename unknown>:0 
  at System.Web.HttpApplication+<Pipeline>c__Iterator1.MoveNext () [0x00000] in <filename unknown>:0 
  at System.Web.HttpApplication.Tick () [0x00000] in <filename unknown>:0

I deploy this exact code to Nginx 1.7.9, Mono 3.12, Mono-FastCGI-Server4 3.0.0 and it works fine. The [WebMethod] fires fine so the only outlier is the HyperFastCGI process. How can I resolve this problem?

<configuration>
    <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
        <!-- Host factory defines how host will be created. SystemWebHostFactory creates host in AppDomain in standard ASP.NET way --> 
        <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
        <!-- <threads> creates threads at startup. Value "0" means default value --> 
        <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
        <!--- Sets the application host root directory -->
        <root-dir>/path/to/website</root-dir>
    </server>

    <listener type="HyperFastCgi.Listeners.NativeListener">
        <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
            <multithreading>Single</multithreading>
        </apphost-transport>
        <protocol>InterNetwork</protocol>
        <address>127.0.0.1</address>
        <port>9000</port>
    </listener>

    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
        <log level="Debug" write-to-console="true" />
        <add-trailing-slash>false</add-trailing-slash>
    </apphost>
</configuration>

Split to separate FastCGI server and application host implementations

As I've already proposed in mono-devel it would be nice to have an ASP.NET host implementation with plugable HTTP-server support. It also a nice thing to have a separate FastCGI-server for serving raw requests for high-performance tasks.

What's the correct way to build with Mono 4?

When running ./autogen.sh on a Linux system equipped with Mono4, the following error occurs:

checking for gmcs... no
configure: error: You need to install 'gmcs'

That's okay because *mcs is deprecated in Mono 4.
I can see in the makefiles that building with Mono 4 (xbuild to receive mono-server-hyperfastcgi4.exe) is supported, but what is the right way to trigger a Mono 4 only build with ./autogen.sh - instead of manually issuing xbuild and taking care of the rest which the build+install scripts do automatically?

(Oh, and I'm on the v0.3 stable branch).

Thanks.

NetworkConnector.SendOutput

I am using ubuntu 13 and latest mono and latest HyperFastCgi
/usr/local/bin/mono-server-hyperfastcgi4 --verbose --appconfigdir /etc/init.d/mono-fastcgi/ /socket=unix:/var/run/mono3-fcgi-0.sock /logfile=/var/log/mono/fastcgi.log /keepalive=false /minthreads=20,4,

I getting this error when i using ab: ab -c 10 -n 50000 http://192.168.2.73/api/json/reply/Hello

Unhandled exception: System.NullReferenceException: Object reference not set to an instance of an object
at Mono.WebServer.HyperFastCgi.NetworkConnector.SendOutput (UInt16 requestId, System.Byte[] data, Int32 length) [0x00000] in :0
at Mono.WebServer.HyperFastCgi.WorkerRequest.SendResponseFromMemory (System.Byte[] data, Int32 length) [0x00000] in :0
at Mono.WebServer.MonoWorkerRequest.ProcessRequest () [0x00000] in :0

Can somebody give me a full example?

Now, My dir tree is as following:

[guotie@linux-188-235 monoTest]$ pwd
/home/guotie/monoTest/monoTest

26 directories, 60 files

And com.config is :

[guotie@linux-188-235 monoTest]$ cat com.config

 <configuration>
       <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
             <!-- Host factory defines how host will be created. SystemWebHostFactory creates host in AppDomain in standard ASP.NET way --> 
            <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
            <!-- <threads> creates threads at startup. Value "0" means default value --> 
            <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
            <!--- Sets the application host root directory -->
      </server>

  <listener type="HyperFastCgi.Listeners.ManagedFastCgiListener">
            <listener-transport type="HyperFastCgi.Transports.CombinedFastCgiListenerTransport"></listener-transport>
            <apphost-transport type="HyperFastCgi.Transports.CombinedAppHostTransport"></apphost-transport>
            <protocol>InterNetwork</protocol>
        <address>0.0.0.0</address>
            <port>9000</port>
</listener>

<apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
            <log level="All" write-to-console="true" />
            <add-trailing-slash>false</add-trailing-slash>
</apphost>
<web-application>
    <name>Root</name>
    <vpath>/</vpath>
    <path>/home/guotie/monoTest/monoTest/</path>
<!-- vhost and port are ignored in xsp.exe -->
    <vhost>*</vhost>
    <vport>9000</vport>
<!-- <enabled> is true by default -->
<!-- <enabled>false</enabled> -->
</web-application>

</configuration>

When i run hyperfastcgi:
[guotie@linux-188-235 monoTest]$ hyperfastcgi4 /config=./com.config /printlog /verbose /loglevels=All
[2015-10-28 01:38:14Z] Debug HyperFastCgi
[2015-10-28 01:38:14Z] Debug Threadpool minw=40,minio=4,maxw=100,maxio=4
[2015-10-28 01:38:14Z] Debug Root directory: /home/guotie/monoTest/monoTest
host-list.c:38: register_host(): *:9000:/:/home/guotie/monoTest/monoTest/ host=0x7f94683e4d90 pinned_host=0x7f94683e4d90 domain=0x13e6770
[2015-10-28 01:38:14Z] Debug Configured host in domain 83694651, id=1
[2015-10-28 01:38:14Z] Debug Listening on port: 9000
[2015-10-28 01:38:14Z] Debug Listening on address: 0.0.0.0
[2015-10-28 01:38:14Z] Debug Application started

But I use chrome visit localhost:9000/, there is no response?

where is the problem?

Redirect /aaa to /aaa/ and /aaa/bbb to /aaa/bbb/

With fastcgi-mono-server, you get redirected from /aaa to /aaa/
code : https://github.com/mono/xsp/blob/f79fc52f1f3825f31fabeb71c908e3f5784cb187/src/Mono.WebServer.FastCgi/ApplicationHost.cs

With HyperFastCgi, we don't get redirected, even though some code for it seems to be present:
https://github.com/xplicit/HyperFastCgi/blob/04caa9d5fea55b5a91dd54e8fcda9769d61011e7/src/Mono.WebServer.HyperFastCgi/WebServer/VPathToHost.cs

Thanks in advance

"Sent unsupported FastCGI protocol version" error with keepalive

I was testing my ServiceStack web service with httperf tool and periodically was getting "sent unsupported FastCGI protocol version" error. Here my error log:

2016/05/04 20:27:12 [error] 23191#0: *18 upstream sent unsupported FastCGI proto
col version: 0 while reading upstream, client: 195.19.44.143, server: localhost:
8000, request: "GET /db/api/thread/list?forum=hgezdt6ufw&limit=52&order=desc&thr
ead=374 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"
2016/05/04 20:27:18 [error] 23191#0: *25 upstream sent unsupported FastCGI proto
col version: 0 while reading upstream, client: 195.19.44.143, server: localhost:
8000, request: "GET /db/api/post/list?post=976184&limit=75&order=asc&forum=qz72 
HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"
2016/05/04 20:27:19 [error] 23191#0: *26 upstream sent unsupported FastCGI proto
col version: 0 while reading upstream, client: 195.19.44.143, server: localhost:
8000, request: "GET /db/api/thread/listPosts?limit=66&order=desc&thread=5651 HTT
P/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"
2016/05/04 20:27:26 [error] 23191#0: *39 upstream sent unsupported FastCGI proto
col version: 0 while reading upstream, client: 195.19.44.143, server: localhost:
8000, request: "GET /db/api/forum/listThreads?related=user&related=forum&limit=3
9&order=asc&forum=mvsf6q HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: 
"127.0.0.1"
2016/05/04 20:27:32 [error] 23191#0: *49 upstream sent unsupported FastCGI proto
col version: 0 while reading upstream, client: 195.19.44.143, server: localhost:
8000, request: "GET /db/api/thread/listPosts?limit=75&order=desc&thread=1556 HTT
P/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "127.0.0.1"

After some research I've found what appearing of the error depends on enabled or disabled keepalive.

Here my Nginx config:

upstream fastcgi_backend {
  server 127.0.0.1:9000;
  #keepalive 32;
}
server {
  listen 8000;
  server_name localhost:8000;
  error_log /var/log/nginx/Forum.Error.log;

  location / {
    root /root/forum/Forum/;
    index index.html index.htm default.aspx Default.aspx;
    fastcgi_index Default.aspx;
    #fastcgi_keep_conn on;
    fastcgi_pass fastcgi_backend;
    include /etc/nginx/fastcgi_params;
  }
}

So, if I will remove # before commented lines I will get this errors. Otherwise, running with --keepalive=false switch everything is alright.

If you need some additional information, I'll be glad provide it for you.

Requesting Help with Errors in HyperFastCgi Log running in Mono4

Hi @xplicit, hope you are well.

My associates and I have been using HFC fairly successfully for a few weeks now. We're currently evaluating Mono4 with a hope to upgrade to it soon. While doing this we have been exercising our stack (in development) under some relatively light load. Unfortunately, we've encountered some issues during our testing.

These errors have appeared in the HyperFastCgi log:

Socket errors/remote host disconnects:

[2015-06-03 03:45:31Z] Debug   Listening on address: /tmp/gameplay
 [0;1mlibev.c:322: cmd_error(): A socket error (0x21) occurred on fd 19.
 [0m [0;1mlibev.c:322: cmd_error(): A socket error (0x21) occurred on fd 19.
 [0m [0;1mlibev.c:322: cmd_error(): A socket error (0x21) occurred on fd 25.
<snip>
 [0m [0;1mlibev.c:322: cmd_error(): A socket error (0x21) occurred on fd 30.
 [0mhost-list.c:38: register_host():    microservice-lt.wcra.mycompany.com:80:/gameplay:/opt/mycompany/microservices/current/Services/Gameplay/Gameplay.Web host=0x7f408abd4958 pinned_host=0x7f408abd4958 domain=0x212f2f0
libev.c:461: Listen():  libevent version: 2.0.16-stable
libev.c:475: Listen():  libevent is using epoll for events.
libev.c:317: cmd_error():   Remote host disconnected from fd 31.
libev.c:317: cmd_error():   Remote host disconnected from fd 94.
libev.c:317: cmd_error():   Remote host disconnected from fd 26.
<snip>

SIGSEGV in native code:

Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) HyperFastCgi.Transports.NativeTransport.EndRequest (HyperFastCgi.Transports.NativeTransport,ulong,int,int) <IL 0x0001c, 0xffffffff>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.CloseConnection () <IL 0x0002c, 0x00058>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.FlushResponse (bool) <IL 0x00007, 0x00024>
  at System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <IL 0x0007b, 0x000d0>
  at (wrapper remoting-invoke-with-check) System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <IL 0x0003a, 0xffffffff>
  at System.Web.HttpResponse.Flush (bool) <IL 0x00136, 0x00227>
  at System.Web.HttpApplication.OutputPage () <IL 0x0001c, 0x0008e>
  at System.Web.HttpApplication.PipelineDone () <IL 0x0003b, 0x0011f>
  at System.Web.HttpApplication/<Pipeline>c__Iterator1.MoveNext () <IL 0x01438, 0x0661c>
  at System.Web.HttpApplication.Tick () <IL 0x00006, 0x00051>
  at System.Web.HttpApplication.Start (object) <IL 0x000aa, 0x0024f>
  at System.Web.HttpApplication.System.Web.IHttpHandler.ProcessRequest (System.Web.HttpContext) <IL 0x0001c, 0x00073>
  at System.Web.HttpRuntime.Process (System.Web.HttpWorkerRequest) <IL 0x000c8, 0x00281>
  at System.Web.HttpRuntime.RealProcessRequest (object) <IL 0x00029, 0x00097>
  at System.Web.HttpRuntime.ProcessRequest (System.Web.HttpWorkerRequest) <IL 0x0002c, 0x0005d>
  at HyperFastCgi.AppHosts.AspNet.MonoWorkerRequest.ProcessRequest () <IL 0x0000a, 0x0009f>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.Process (HyperFastCgi.Interfaces.IWebResponse) <IL 0x00001, 0x00014>
  at HyperFastCgi.Transports.BaseAppHostTransport/<AddBodyPart>c__AnonStorey0.<>m__1 (object) <IL 0x00011, 0x000bb>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00062, 0xffffffff>

Native stacktrace:

    /usr/bin/mono() [0x4b1d6c]
    /usr/bin/mono() [0x50833e]
    /usr/bin/mono() [0x428bfd]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f565e0cdcb0]
    /usr/lib/libevent-2.0.so.5(evbuffer_add+0x67) [0x7f5659897e67]
    /usr/lib/libhfc-native.so(+0x5742) [0x7f5659fc5742]
    /usr/lib/libhfc-native.so(end_request+0x132) [0x7f5659fc5a92]
    [0x40e7305c]

Debug info from gdb:

host-list.c:38: register_host():    microservice-lt.wcra.mycompany.com:80:/gameplay:/opt/mycompany/microservices/current/Services/Gameplay/Gameplay.Web host=0x7f565d3d4c28 pinned_host=0x7f565d3d4c28 domain=0x1acb5a0
libev.c:461: Listen():  libevent version: 2.0.16-stable
libev.c:475: Listen():  libevent is using epoll for events.
libev.c:317: cmd_error():   Remote host disconnected from fd 33.
libev.c:317: cmd_error():   Remote host disconnected from fd 23.
libev.c:317: cmd_error():   Remote host disconnected from fd 24.
libev.c:317: cmd_error():   Remote host disconnected from fd 51.

=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

SIGABRT in native code:

mono: pthread_mutex_lock.c:84: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed.
host-list.c:38: register_host():    microservice-lt.wcra.mycompany.com:80:/gameplay:/opt/mycompany/microservices/current/Services/Gameplay/Gameplay.Web host=0x7fbc86fd4c28 pinned_host=0x7fbc86fd4c28 domain=0x1b406d0
libev.c:461: Listen():  libevent version: 2.0.16-stable
libev.c:475: Listen():  libevent is using epoll for events.
libev.c:317: cmd_error():   Remote host disconnected from fd 15.
libev.c:317: cmd_error():   Remote host disconnected from fd 48.
libev.c:317: cmd_error():   Remote host disconnected from fd 46.
libev.c:317: cmd_error():   Remote host disconnected from fd 29.
libev.c:317: cmd_error():   Remote host disconnected from fd 16.
libev.c:317: cmd_error():   Remote host disconnected from fd 53.
libev.c:317: cmd_error():   Remote host disconnected from fd 74.
libev.c:317: cmd_error():   Remote host disconnected from fd 45.
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) HyperFastCgi.Transports.NativeTransport.EndRequest (HyperFastCgi.Transports.NativeTransport,ulong,int,int) <IL 0x0001c, 0xffffffff>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.CloseConnection () <IL 0x0002c, 0x00058>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.FlushResponse (bool) <IL 0x00007, 0x00024>
  at System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <IL 0x0007b, 0x000d0>
  at (wrapper remoting-invoke-with-check) System.Web.HttpResponseStream.Flush (System.Web.HttpWorkerRequest,bool) <IL 0x0003a, 0xffffffff>
  at System.Web.HttpResponse.Flush (bool) <IL 0x00136, 0x00227>
  at System.Web.HttpApplication.OutputPage () <IL 0x0001c, 0x0008e>
  at System.Web.HttpApplication.PipelineDone () <IL 0x0003b, 0x0011f>
  at System.Web.HttpApplication/<Pipeline>c__Iterator1.MoveNext () <IL 0x01438, 0x0661c>
  at System.Web.HttpApplication.Tick () <IL 0x00006, 0x00051>
  at System.Web.HttpApplication.Start (object) <IL 0x000aa, 0x0024f>
  at System.Web.HttpApplication.System.Web.IHttpHandler.ProcessRequest (System.Web.HttpContext) <IL 0x0001c, 0x00073>
  at System.Web.HttpRuntime.Process (System.Web.HttpWorkerRequest) <IL 0x000c8, 0x00281>
  at System.Web.HttpRuntime.RealProcessRequest (object) <IL 0x00029, 0x00097>
  at System.Web.HttpRuntime.ProcessRequest (System.Web.HttpWorkerRequest) <IL 0x0002c, 0x0005d>
  at HyperFastCgi.AppHosts.AspNet.MonoWorkerRequest.ProcessRequest () <IL 0x0000a, 0x0009f>
  at HyperFastCgi.AppHosts.AspNet.AspNetNativeWebRequest.Process (HyperFastCgi.Interfaces.IWebResponse) <IL 0x00001, 0x00014>
  at HyperFastCgi.Transports.BaseAppHostTransport/<AddBodyPart>c__AnonStorey0.<>m__1 (object) <IL 0x00011, 0x000bb>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <IL 0x00062, 0xffffffff>

Native stacktrace:

    /usr/bin/mono() [0x4b1d6c]
    /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7fbc87ccfcb0]
    /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7fbc879380d5]
    /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7fbc8793b83b]
    /lib/x86_64-linux-gnu/libc.so.6(+0x2ed9e) [0x7fbc87930d9e]
    /lib/x86_64-linux-gnu/libc.so.6(+0x2ee42) [0x7fbc87930e42]
    /lib/x86_64-linux-gnu/libpthread.so.0(pthread_mutex_lock+0x184) [0x7fbc87cca004]
    /usr/lib/libhfc-native.so(+0x5732) [0x7fbc5f9f8732]
    /usr/lib/libhfc-native.so(end_request+0x132) [0x7fbc5f9f8a92]
    [0x41248f8c]

Debug info from gdb:


=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

We're not sure whether these are related, or independent, but they seem to happen during the same tests.

We can reproduce these errors fairly reliably. We've noticed the following patterns:

The errors happen only on Mono4. Mono3 does not experience them.
The errors happen after a short period of running under light to moderate load, between 60 - 200 requests/second. Within 3-5 minutes after starting the test, the errors start to occur.
The HFC process often does not die when this happens. However, not sure how frequently that occurs.
This happens only when using our microservice framework. When we test using a trivial HttpHandler-based test service it doesn't happen. Of course, this could be caused by a bug in our framework, or because our framework is executing many more code paths than the trivial app.
The errors seem to be associated with the user of an elastic load balancer in front of our services. We need to do more to isolate that case completely in order to be certain, but it seems to correlate.

I've noticed that both of the native stacks above involve libpthread, but i can't tell whether this is related to a real threading issue, or if it's just a coincidence.

I've tried to research these errors by searching your GitHub site and elsewhere. I know the socket errors were discussed in #24 and are expected to be fixed in 9c14b50. We are using b0fcd0b from May 2, so we have that fix already.

We are grateful for any suggestions you might have that could help us with debugging this problem.

Thanks in advance, and kind regards, Matt

Getting SIGABRTs

Hi,
We're using supervisord which keeps mono-server-hyperfastcgi4 running, using the most current revision of HFC and Mono 4.2.1.
Our log shows that HFC crashed from time to time due to SIGABRT, "exit status 1" and SIGSEV.

That's the log:

2016-01-20 19:12:27,622 INFO exited: mono-server-hyperfastcgi4 (terminated by SIGABRT (core dumped); not expected)

2016-01-20 19:12:28,624 INFO spawned: 'mono-server-hyperfastcgi4' with pid 3581
2016-01-20 19:12:30,459 INFO success: mono-server-hyperfastcgi4 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-01-20 19:12:38,284 INFO exited: mono-server-hyperfastcgi4 (terminated by SIGABRT (core dumped); not expected)

2016-01-20 19:12:39,285 INFO spawned: 'mono-server-hyperfastcgi4' with pid 3662
2016-01-20 19:12:40,478 INFO success: mono-server-hyperfastcgi4 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-01-20 21:14:26,362 INFO exited: mono-server-hyperfastcgi4 (terminated by SIGABRT (core dumped); not expected)

2016-01-20 21:14:27,365 INFO spawned: 'mono-server-hyperfastcgi4' with pid 3874
2016-01-20 21:14:29,171 INFO success: mono-server-hyperfastcgi4 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-01-21 02:09:13,084 INFO exited: mono-server-hyperfastcgi4 (exit status 1; not expected)

2016-01-21 02:09:14,087 INFO spawned: 'mono-server-hyperfastcgi4' with pid 4259
2016-01-21 02:09:15,599 INFO success: mono-server-hyperfastcgi4 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-01-21 03:41:36,285 INFO exited: mono-server-hyperfastcgi4 (terminated by SIGABRT (core dumped); not expected)

2016-01-21 14:38:16,069 INFO spawned: 'mono-server-hyperfastcgi4' with pid 5281
2016-01-21 14:38:17,867 INFO success: mono-server-hyperfastcgi4 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-01-21 16:15:30,754 INFO exited: mono-server-hyperfastcgi4 (terminated by SIGSEGV (core dumped); not expected)

2016-01-21 16:15:31,856 INFO spawned: 'mono-server-hyperfastcgi4' with pid 5498

Currently I'm trying to attach gdb to catch the SIGABRT signal ((gdb) catch signal SIGABRT) to hopefully get a stacktrace. For that I have to generate debug symbols for the libnative.

Any hints what the reason of this could be, or how to find out what the reason is? Any help is appreciated, even if it is generic debugging help.

Thanks!

License file is empty

Build on Mono 4.0

I upgraded mono to 4.0 because there are a few bugs fixed there which help me. Problem is that it drops support for .net 2.0 and 4.0 and (at least I think that's the reason) I cannot compile HFCGI because there is no gmcs (running ./autogen.sh --prefix=/usr reports that). Here are the release notes: http://www.mono-project.com/docs/about-mono/releases/4.0.0/

Any hints how to build HFCGI with mono 4.0?

Native listener dropping concurrent requests on unix socket

I am seeing issues with Apache Bench at concurrency greater than 10 threads. I've recreated this on Ubuntu 14.04.1 LTS, mono 4.0.2 and nginx 1.4.6 to eliminate any quirks from our environment.

Native listener using unix sockets

ab -t60 -n 1500 -c 10 'http://10.100.125.94/admin/status'
Failed requests:        2
Non-2xx responses:      2
Requests per second:    665.51 [#/sec] (mean)

ab -t60 -n 1500 -c 30 'http://10.100.125.94/admin/status'
Failed requests:        534
Non-2xx responses:      534
Requests per second:    1595.88 [#/sec] (mean)

ab -t60 -n 1500 -c 100 'http://10.100.125.94/admin/status'
Failed requests:        547
Non-2xx responses:      955
Requests per second:    1831.00 [#/sec] (mean)

Running the managed code I can get currency of 100-200 without issue. There is a reduction in performance but it doesn't drop requests resulting in 502 responses from nginx.

Your blog showed you at least achieving 30 concurrent connections without issue. Did you see these issues?

ManagedFastCgiListener not closing sockets

Hello. Great work with the project. Unfortunately I am having issues with the ManagedFastCgiListener under CentOS 6.

Have you seen this problem where many sockets are not closing?

I can reproduce with ab -t60 -n 50000 -c 50 'http://somehost' and watching the sockets/file descriptors climb (netstat -ap | grep mono | wc -l and ls -l /proc/pidof mono/fd) until hyperfastcgi hits the ulimit and crashes.

unix  3      [ ]         STREAM     CONNECTED     1967223 21956/mono          /tmp/hyperfastcgi.socket
unix  3      [ ]         STREAM     CONNECTED     1967221 21956/mono          /tmp/hyperfastcgi.socket
unix  3      [ ]         STREAM     CONNECTED     1967219 21956/mono          /tmp/hyperfastcgi.socket
unix  2      [ ]         STREAM     CONNECTED     1967215 21956/mono          /tmp/hyperfastcgi.socket
unix  2      [ ]         STREAM     CONNECTED     1967209 21956/mono          /tmp/hyperfastcgi.socket
unix  3      [ ]         STREAM     CONNECTED     1966980 21956/mono          /tmp/hyperfastcgi.socket

hyperfastcgi

<configuration>
    <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
        <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
        <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
    </server>
    <listener type="HyperFastCgi.Listeners.ManagedFastCgiListener">
        <listener-transport type="HyperFastCgi.Transports.ManagedFastCgiListenerTransport"></listener-transport>
        <apphost-transport type="HyperFastCgi.Transports.ManagedAppHostTransport"></apphost-transport>
        <protocol>Unix</protocol>
        <address>/tmp/hyperfastcgi.socket</address>
    </listener>
    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
        <log level="Debug" write-to-console="true" />
        <add-trailing-slash>false</add-trailing-slash>
    </apphost>
    <web-applications>
        <web-application>
            <name>website</name>
            <vhost></vhost>
            <vport>80</vport>
            <vpath>/</vpath>
            <path>/var/www/website</path>
        </web-application>
    </web-applications>
</configuration>

nginx

upstream hyperfastcgi_backend {
  server unix:/tmp/hyperfastcgi.socket;
  keepalive 32;
}
server {
  listen 80;

  client_max_body_size 50M;
#  keepalive_timeout 5;

  access_log /var/log/nginx/access.log;

  location / {
#    proxy_read_timeout 600;
    root /var/www/website;
    index default.aspx Default.aspx;
    fastcgi_index Default.aspx;
    fastcgi_keep_conn on;
    fastcgi_pass hyperfastcgi_backend;
    include /etc/nginx/fastcgi_params;
  }
}

mono is a custom build of 3.2 which is not the newest so I haven't ruled that out yet.

v0.3_stable always times out, not responding

I've downloaded & built v0.3_stable branch, ran it like this mono-server-hyperfastcgi4 /applications=/:/home/...../CRM.WebHost/ /socket=tcp:127.0.0.1:9000
it does start properly (according to the application log) however I cannot complete any request.
According to the nginx log, every request times out
18405#0: *5206 upstream timed out (110: Connection timed out) while reading upstream, client: <my up>, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "< my hostname>"

When I simply run default fastcgi-mono-server4 instead (with the same configuration), it does work properly and no FastCGI request times out

I should mention that my requests are being processed in hundreds of milliseconds, always less than a second, so they're not anywhere near the timeout limit.

Mono version

Mono JIT compiler version 3.12.1 (tarball Fri Mar  6 19:12:47 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

Error: Missing method .ctor in assembly

Hi,

I have followed the instructions for setting up HyperfastCgi and configured Nginx to proxy requests to it.

However when I browse to the site I get the following error:
Missing method .ctor in assembly /tmp/root-temp-aspnet-0/e8de8a00/assembly/shadow/360ac660/77395a79_41339c19_00000001/site.dll, type System.Runtime.CompilerServices.ExtensionAttribute
Can't find custom attr constructor image: /tmp/root-temp-aspnet-0/e8de8a00/assembly/shadow/360ac660/77395a79_41339c19_00000001/site.dll mtoken: 0x0a00040b

Assertion at class.c:5597, condition `!mono_loader_get_last_error ()' not met

And the HyperfastCgi process terminates.

Do you know if this is a bug, or a misconfiguration?

Many thanks.

Adam

Aborted after a binary file changed

Change: NuGetServer.dll
Stacktrace:


Native stacktrace:

        /usr/bin/mono() [0x4b73d8]
        /usr/bin/mono() [0x50f13b]
        /usr/bin/mono() [0x423d22]
        /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340) [0x7f8b6488c340]
        /lib/x86_64-linux-gnu/libc.so.6(+0x870c6) [0x7f8b6453e0c6]
        /usr/lib/libhfc-native.so(unregister_host+0x84) [0x7f8b27bfac74]
        [0x419ae9a6]

Debug info from gdb:

Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No threads.

=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

Masterpages not working? (hyperfastcgi 2 + nginx + unix socket)

I seem to have run into a problem with HFC and compiled master page cache. I am using the latest trunk (compiled in docker vs mono 3.12) which is exactly equal to the current v0.3 stable.

When I load my site-with-masterpages with HFC, the first page loaded displays correctly. The remaining pages show correct titles and updated header fields, but every page loaded displays the same content for the asp:Content chunk. (If the first page loaded is a redirect after login, the app goes into a redirect loop because every response is the same as the initial redirect.) I have snooped the socket using socat and the request is definitely for the correct page, while the response is as I describe.

Another thing that seems to indicate a caching issue is that if I create new pages by "cp index.aspx index2.aspx", the first copy I create loads correctly, and then more copies that I create sometimes display the same as the first copied page, and sometimes do not.

Site has been working in IIS and xsp2 for years. Works in mono-fastcgi-server2 using the same running instance of nginx (no changes to the nginx configuration when comparing mono-fastcgi vs hyperfastcgi), Between that and the socat snooping, I do not think there is any issue at the nginx level.

I will try to reproduce the problem with a minimal example.

HyperFastCgi ThreadPool'ed NativeListener does not work with the new Mono 4.2.1 release

I'm running HyperFastCGI in an isolated Docker container based on Debian Jessie, and just switched Mono from version 4.0 to 4.2.1.102.

With 4.2.1 the ASP.NET application does not come up / does not get initialized. Requests end in a timeout.

Nothing else except updating to Mono 4.2.1.102 has been changed.

HyperFastCGI is built from the latest commit in trunk (9f8a4df).

I don't have any more informations right now, but will continue researching on this.

/Edit: It's the NativeListener which is not working. When switching to the ManagedFastCgiListener the ASP.NET app gets called and responds.

Dummy socket connections never closed by ManagedFastCgiListener

I'm not sure if you will want to fix this but by using a Monit socket health check HyperFastCgi was leaving socket connections open.

In monit you can do this

  if failed unixsocket /tmp/hyperfastcgi.socket
    then restart

Or with netcat do

nc -U /tmp/hyperfastcgi.socket

And because neither is valid Fast CGI, or send any data at all, the connection is never closed

ll /proc/`pidof mono`/fd/ | grep socket | wc -l
netstat -ap | grep mono | wc -l

I tried to dig further with an intermediate socket using socat... but couldn't spot anything http://pastebin.com/XacKRCCK

 socat -t2 -d -d -d -d -x -v UNIX-LISTEN:/tmp/test.socket,mode=777,reuseaddr,fork UNIX-CONNECT:/tmp/hyperfastcgi.socket

The native listener did not have this problem. Excluding this monit check resolves my problem so feel free to close this issue if you don't feel it is worth fixing.

asp:ScriptManager not working

Alt Linux 7.0.5 - Nginx 1.7.11 - Mono 3.12.1 - last HyperFastCGI from sources

Test.aspx page:

<%@ Page Language="C#" AutoEventWireup="true" %>
<!DOCTYPE html>
<html>
<head>
    <title>Test</title>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
</head>
<body>
    <form id="form1" runat="server">
    <asp:ScriptManager ID="ScriptManager1" runat="server" ScriptMode="Release" />
    </form>
</body>
</html>

Nginx config file:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    upstream fastcgi_backend {
    server 127.0.0.1:9000;
    keepalive 32;
    }

    server {
        listen       80;
        server_name  foxtrot;
    access_log off;

    location / {
        root /var/www/abc;
        index index.html index.htm default.aspx Default.aspx;
        fastcgi_index default.aspx;
        fastcgi_keep_conn on;
        fastcgi_pass fastcgi_backend;
        include /usr/local/nginx/conf/fastcgi_params;
    }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
}

fastcgi_params:

fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
fastcgi_param  HTTPS              $https if_not_empty;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param  REDIRECT_STATUS    200;

fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

hyperfastcgi config

<configuration>
    <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
        <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
        <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
    </server>

<listener type="HyperFastCgi.Listeners.NativeListener">
    <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
        <multithreading>ThreadPool</multithreading>
    </apphost-transport>
        <protocol>InterNetwork</protocol>
        <address>127.0.0.1</address>
        <port>9000</port>
    </listener>

    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
        <log level="Debug" write-to-console="true" />
        <add-trailing-slash>false</add-trailing-slash>
    </apphost>
    <web-applications>
        <web-application>
            <name>ABC</name>
            <vhost>ABC</vhost>
            <vport>80</vport>
            <vpath>/</vpath>
            <path>/var/www/abc</path>
        </web-application>
    </web-applications>
</configuration>

Test.aspx renders three links to axd resources with complex parameters:

<script src="/WebResource.axd?d=LKGCVHDwbseBVJfJ%2bQn7fIVXz1Bn%2fNg%2bemO8YM1At1U%3d_aQwKQWrW%2f582vFXKpmpzuwinW6V3jVvriN8RKXPP8II%3d_f&t=635632280460000000" type="text/javascript"></script>

<script src="/ScriptResource.axd?d=piCBLGDbkDrbFkdp9ycaAcRHu6EPEytfQxBt0GhV6P8%3d_LI5vlQ2pwlWtSue0sJ8qvbP2B6BTqY%2bon2vwV8KDjOg%3df_f&t=635632280840000000&n=f" type="text/javascript"></script>
<script src="/ScriptResource.axd?d=piCBLGDbkDrbFkdp9ycaAcRHu6EPEytfQxBt0GhV6P8%3d_qTgqZkpFUWS9rkpIomLBFiZH0Iud2jsfuovyJemnuSs%3df_f&t=635632280840000000&n=f" type="text/javascript"></script>

The first ScriptResource.axd link (MicrosoftAjax.js) does not work with the error (in browser) "The network connection was lost".

HyperFastCGI log in most cases:

hyperfastcgi4 /config=/usr/local/etc/hyperfastcgi/server.config /printlog /logevents=All
[2015-03-31 20:58:42Z] Debug   Register native transport
host-list.c:38: register_host():    ABC:80:/:/var/www/abc host=0x7f319e7fa678 pinned_host=0x7f319e7fa678 domain=0xcbed00
libev.c:461: Listen():  libevent version: 2.0.21-stable-dev
libev.c:475: Listen():  libevent is using epoll for events.
libev.c:322: cmd_error(): A socket error (0x21) occurred on fd 14.
libev.c:180: shutdown_cmdsocket(): Error shutting down client connection on fd 14: 107 (Конечная точка передачи не подсоединена)

sometimes

libev.c:317: cmd_error(): Remote host disconnected from fd 16.

Nginx log:

2015/03/31 19:56:35 [error] 10142#0: *235 upstream sent unsupported FastCGI protocol version: 120 while reading upstream, client: 127.0.0.1, server: foxtrot, request: "GET /ScriptResource.axd?d=jC0hhCq2LjyHJpEqYfYa0r2Ack4y24phTbtK60bsvxM%3d_XQCGSaSmnofCnl64zTWM5iPx4rZ7Ge3i3wFMMBl%2f3ac%3df_f&t=635632280840000000&n=f HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "localhost", referrer: "http://localhost/test.aspx"

If I run Test.aspx under xsp4, all works fine.

mono-server-hyperfastcgi2 does not load the correct system assemblies

When running mono-server-hyperfastcgi2 for .NET 2.0/3.5 projects the following error is thrown:

root@app08:/src/HyperFastCgi# mono-server-hyperfastcgi2 /applications=/test/:/var/www/test/ /socket=tcp:127.0.0.1:9002 /verbose /keepalive=false /addtrailingslash=true
Adding applications '/test/:/var/www/test/'...
Registering application:
    Host:          any
    Port:          any
    Virtual path:  /test/
    Physical path: /var/www/test/
Missing method System.Threading.Monitor::Enter(object,bool&) in assembly /usr/local/lib/mono/2.0/mscorlib.dll, referenced in assembly /usr/local/lib/mono/gac/Mono.WebServer.HyperFastCgi/1.0.5283.19812__0738eb9f132ed756/Mono.WebServer.HyperFastCgi.exe

Unhandled Exception:
System.MissingMethodException: Method not found: 'System.Threading.Monitor.Enter'.

Server stack trace:
  at Mono.WebServer.HyperFastCgi.Server.Start (GeneralSocketType sockType, System.String address, Int32 port, Boolean keepAlive, Boolean useThreadPool) [0x00000] in <filename unknown>:0
  at Mono.WebServer.HyperFastCgi.ApplicationHost.Start (GeneralSocketType sockType, System.String address, Int32 port, Boolean keepAlive, Boolean useThreadPool) [0x00000] in <filename unknown>:0
  at (wrapper managed-to-native) System.Runtime.Remoting.RemotingServices:InternalExecute (System.Reflection.MethodBase,object,object[],object[]&)
  at System.Runtime.Remoting.RemotingServices.InternalExecuteMessage (System.MarshalByRefObject target, IMethodCallMessage reqMsg) [0x00000] in <filename unknown>:0

This appears to be because the v4.0 assemblies are mixed up with the v2.0 assemblies.

Provide sample init.d auto-run with automatic .webapp read.

It would be nice to have some init.d to autorun service, with options to run under www-data, automatic reload after dir of webapp's changes, one process per webapp file (similar iis pool config).

Remove my email from Wiki -> Nginx configuration page

Keep working on it

I run ServiceStack (v4 now) and I want to run it as fast as possible on linux. You have something here!

Better explanation of <root-dir>

Is this server.config option for searching for .webapps ? Or is this just a shortcut for a webapp configured to '/'?

Question: Use Socket Async API for better performance

Hey great job so far on this implementation. We definitely need something better than the default mono FastCgi implementation to be able to use Mono for web hosting outside of Windows.

I was doing some benchmarking on HyperFastCgi vs evhttp-sharp and HyperFastCgi was only about 25% slower than evhttp-sharp which is pretty impressive considering it's using all C# and native mono whereas evhttp is delegating a lot of the work to libev.

I was looking at the code a bit and I noticed the socket receive code performance could be improved. In particular, you could be using the SendAsync/ReceiveAsync calls in order to reduce the pressure on the GC. I'm not sure if it will make a tremendous amount of difference but those calls reuse the same buffers which makes the overhead very low. It's a bit of a challenge to use those APIs out of the box it but there is a project which gives a nice wrapper around them here: https://github.com/safakgur/Dawn.SocketAwaitable

( Also, we've been having some discussions on the official mono devel mailing list about all the different solutions out there and how to get the best performance. )

The async APi would look something like this:

// This would be shared for all socket operations.. It's a single memory buffer that gets split up 
// for each socket operation. First arg is size of each buffer and 2nd is number of buffers.
var bufferManager = new BlockingBufferManager(2048, 100);

// This would also be shared.. It's the SocketAsyncEventArgs pool which also gets pooled to reduce
// pressure on GC. Argument is the initial pool size. It will allocate more if needed.
var socketAwaitablePool = new SocketAwaitablePool(100)

// Listener loop
while (running) {
    SocketAwaitable socketAwaitable = null;
    ArraySegment<byte> buffer = new ArraySegment<byte>();

    try
    {                    
        socketAwaitable = socketAwaitablePool.Take();
        socketAwaitable.Clear();

        buffer = bufferManager.GetBuffer();
        socketAwaitable.Buffer = buffer;

        var result = await listener.AcceptAsync(socketAwaitable);

        if (result != SocketError.Success || socketAwaitable.AcceptSocket == null)
        {
            continue;
        }

        var socket = socketAwaitable.AcceptSocket;
        socket.NoDelay = true;

        ........
    }
    catch(Exception e)
    {
        ........
    }
    finally
    {
        if (socketAwaitable != null)
        {
            socketAwaitable.Clear();
            socketAwaitablePool.Add(socketAwaitable);
        }

        if (buffer.Array != null)
        {
            bufferManager.ReleaseBuffer(buffer);
        }
    }        
}


// To send data on a socket
// packetBytes is a byte[] of the payload

int totalBytesSent = 0;

var socketAwaitable = socketAwaitablePool.Take();

var sendBuffer = bufferManager.GetBuffer();
var sendBufferMaxSize = sendBuffer.Count;

try
{
    while (totalBytesSent < packetBytes.Length)
    {
        socketAwaitable.Clear();
        var writeCount = Math.Min(packetBytes.Length - totalBytesSent, sendBufferMaxSize);
        Buffer.BlockCopy(packetBytes, totalBytesSent, sendBuffer.Array, sendBuffer.Offset, writeCount);

        socketAwaitable.Buffer = new ArraySegment<byte>(sendBuffer.Array, sendBuffer.Offset, writeCount);

        var result = await Socket.SendAsync(socketAwaitable);

        if (result != SocketError.Success || socketAwaitable.Transferred.Count == 0)
        {            
            // failed to send..
        }

        totalBytesSent += socketAwaitable.Transferred.Count;
    }
}
catch(Exception e)
{
    ....
}
finally
{
    socketAwaitable.Clear();
    socketAwaitablePool.Add(socketAwaitable);

    bufferManager.ReleaseBuffer(sendBuffer);
}

Using ThreadPool is slower than Task

I'm still doing performance tests regularly and wanted to let you know that using <multithreading>ThreadPool</multithreading> is performing 20-50% slower in average than <multithreading>Task</multithreading> with our real life user session load test setup.

I had expected that ThreadPool would be actually faster than Task.
What do you think about Task vs. ThreadPool performance comparison?

ASP.NET vNext support

I have seen in your roadmap, that ASP.NET vNext is planned for v0.6, but I thought it would be nice to have an GitHub issue to share some thoughts about it 😄

As far as I understood the whole thing, there are two possibilities to implement an ASP.NET vNext Web-Server. Either the "traditional" way, the Server provides an executable and then calls the Web Application, like IIS does and like HyperFastCgi is currently implemented. But I have not seen any (open) implementation for ASP.NET vNext so far.
The other way is to provide a DLL, which is called by "Microsoft.AspNet.Hosting" and configured in the project.json.

An example of the 2nd implementation is firefly.
For HyperFastCgi I see the following additions:

An implementation of IServerFactory
- The start would not simmilar code than Main
A new application host
- Needs an CallContext which implements IHttpRequestFeature and IHttpResponseFeature
- In ProcessRequest, the delegate passed to IServerFactory must be called, with the CallContext containing the request informations
  ** The call must be awaited and then the values from the response-values from CallContext must be set to the IWebResponse

"Connection lost" between nginx and HyperFastCGI

We encountered this issue already twice:

Request sent to nginx
Nginx should forward this request to HFC.
The request is not reaching Mono (no log output on the mono side), and nginx writes it's infamous HTTP ERROR 499 entry into it's log file: "POST /api/0.0.1/list.json? HTTP/1.1" 499 0 "-" "BestHTTP". This 499 entry is nginx-specific and only kept in the logs, not sent out to the client. The clients request times out.
The HFC log file doesn't have any output on that. We're calling it this way: usr/bin/mono /usr/lib/hyperfastcgi/4.0/HyperFastCgi.exe /config=/hyperfastcgi.conf /applications=/:/usr/aspnet/ /verbose /loglevels=All /logfile=/var/log/nginx/fastcgi.log.

We're using the most current revision of HFC and Mono 4.2.1.

This is our HFC config:

<configuration>
        <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
                <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
                <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
        </server>

        <listener type="HyperFastCgi.Listeners.NativeListener">
                <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
                        <multithreading>ThreadPool</multithreading>
                </apphost-transport>
            <protocol>InterNetwork</protocol>
            <address>127.0.0.1</address>
            <port>9000</port>
        </listener>

    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
                <log level="Debug" write-to-console="true" />
                <add-trailing-slash>false</add-trailing-slash>
    </apphost>

</configuration>

That's all we got from the HFC log:

[2016-01-20 16:29:34Z] Debug   HyperFastCgi
[2016-01-20 16:29:35Z] Debug   Threadpool minw=40,minio=4,maxw=100,maxio=100
[2016-01-20 16:29:35Z] Debug   Root directory: /
[2016-01-20 16:29:36Z] Debug   Listening on port: 9000
[2016-01-20 16:29:36Z] Debug   Listening on address: 127.0.0.1
[2016-01-21 10:24:51Z] Debug   HyperFastCgi
[2016-01-21 10:24:51Z] Debug   Threadpool minw=40,minio=4,maxw=100,maxio=100
[2016-01-21 10:24:51Z] Debug   Root directory: /
[2016-01-21 10:24:52Z] Debug   Listening on port: 9000
[2016-01-21 10:24:52Z] Debug   Listening on address: 127.0.0.1

After restarting HFC every request got served correctly and fast again.
On a sidenote - I had to kill -9 HFC, because a "normal kill" wouldn't work.

The question is - What can we do to investigate this issue further, other than debugging the code in an UI? (because we don't know what the root cause of this behaviour is to reproduce it). Are there any system information or logs which can help telling why HFC obviously stops responding?

System.NullReferenceException at HyperFastCgi.Transports.BaseManagedListenerTransport.AddHeader

Just wanted to let you know about this exception, which happened to us during a stress test.
Havn't yet investigated further, will do as soon as I got time:

root@4c1daf7b56af:/# tail -f -n 500 /var/log/supervisor/mono-server-hyperfastcgi4-stdout---supervisor-cAIE5v.log

[2016-03-11 15:12:24Z] Debug   HyperFastCgi
[2016-03-11 15:12:24Z] Debug   Configured host in domain d95d475a, id=1
log4net:ERROR You have tried to set a null level to root.
log4net.Core.LogException: Error in the application.

Unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object
  at HyperFastCgi.Transports.BaseManagedListenerTransport.AddHeader (System.String name, System.String value, Boolean isHeader, System.Object userData) <0x4018a230 + 0x0005b> in <filename unknown>:0
  at HyperFastCgi.Helpers.FastCgiProtocol.FcgiUtils.ParseParameters (System.Byte[] data, HyperFastCgi.Helpers.FastCgiProtocol.AddHeaderDelegate func, System.Object userData) <0x40189880 + 0x0023b> in <filename unknown>:0
  at HyperFastCgi.Transports.BaseManagedListenerTransport.Process (UInt64 listenerTag, Int32 requestNumber, System.Byte[] header, System.Byte[] recordBody) <0x40184ac0 + 0x006bb> in <filename unknown>:0
  at HyperFastCgi.Listeners.FastCgiNetworkConnector.ProcessRecord (System.Byte[] header, System.Byte[] body) <0x401848e0 + 0x00196> in <filename unknown>:0
  at HyperFastCgi.Listeners.FastCgiNetworkConnector.ReceiveCallback (IAsyncResult ar) <0x40183ec0 + 0x00508> in <filename unknown>:0
[2016-03-11 15:25:22Z] Debug   HyperFastCgi
[2016-03-11 15:25:22Z] Debug   Configured host in domain d95d475a, id=1
log4net:ERROR You have tried to set a null level to root.
log4net.Core.LogException: Error in the application.

Unhandled Exception: System.ApplicationException: Argument "socket" is unknown.

Trying to use HyperFastCgi (v0.3_stable branch) with nginx. When I try running:

sudo mono 4.0/Mono.WebServer.HyperFastCgi.exe /configfile=/usr/lib/hyperfastcgi/server.config /stopable

I receive the following exception and the app crashes:

Unhandled Exception:
System.ApplicationException: Argument "socket" is unknown.
  at Mono.WebServer.HyperFastCgi.ConfigurationManager.get_Item (System.String name) [0x00000] in <filename unknown>:0
  at Mono.WebServer.HyperFastCgi.MainClass.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ApplicationException: Argument "socket" is unknown.
  at Mono.WebServer.HyperFastCgi.ConfigurationManager.get_Item (System.String name) [0x00000] in <filename unknown>:0
  at Mono.WebServer.HyperFastCgi.MainClass.Main (System.String[] args) [0x00000] in <filename unknown>:0

The server.config file is using Unix sockets:

<configuration>
        <server type="HyperFastCgi.ApplicationServers.SimpleApplicationServer">
                <!-- Host factory defines how host will be created. SystemWebHostFactory creates host in AppDomain in standard ASP.NET way -->
                <host-factory>HyperFastCgi.HostFactories.SystemWebHostFactory</host-factory>
                <!-- <threads> creates threads at startup. Value "0" means default value -->
                <threads min-worker="40" max-worker="0" min-io="4" max-io="0" />
                <!--- Sets the application host root directory -->
                <!-- <root-dir>/path/to/your/dir</root-dir> -->
        </server>
        <listener type="HyperFastCgi.Listeners.NativeListener">
                <apphost-transport type="HyperFastCgi.Transports.NativeTransport">
                        <multithreading>Single</multithreading>
                </apphost-transport>
            <protocol>Unix</protocol>
            <address>/tmp/fastcgi.socket</address>
        </listener>
    <apphost type="HyperFastCgi.AppHosts.AspNet.AspNetApplicationHost">
                <log level="Debug" write-to-console="true" />
                <add-trailing-slash>false</add-trailing-slash>
    </apphost>
    <web-applications>
        <web-application>
                <name>MyApp</name>
                <vhost>ssbench3</vhost>
                <vport>81</vport>
                <vpath>/</vpath>
                <path>/var/www/nginx-mono</path>
        </web-application>
    </web-applications>
</configuration>

What am I doing wrong?

System.DllNotFoundException: libhfc-native

I'm getting the following error when I attempt to run:

hyperfastcgi4 /config=/etc/init.d/mono-fastcgi/hyperfastcgi.conf /logfile=/var/log/mono/fastcgi.log &

Unhandled Exception:
System.DllNotFoundException: libhfc-native
  at (wrapper managed-to-native) HyperFastCgi.Listeners.NativeListener:Listen (uint16,string,uint16)
  at HyperFastCgi.Listeners.NativeListener.Listen () [0x00000] in <filename unknown>:0
  at HyperFastCgi.MainClass.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.DllNotFoundException: libhfc-native
  at (wrapper managed-to-native) HyperFastCgi.Listeners.NativeListener:Listen (uint16,string,uint16)
  at HyperFastCgi.Listeners.NativeListener.Listen () [0x00000] in <filename unknown>:0
  at HyperFastCgi.MainClass.Main (System.String[] args) [0x00000] in <filename unknown>:0

How to install newer version?

Question: I have an older version of HFCGI. I'd like to update it with the latest stable. What commands do I run?

Is Asp.Net Mvc projects supported?

Can you add configuration examples for MVC application hosting?

Server timeouts and returns nothing instead of expected result

I am running Ubuntu 14.04 with Mono 3.2.8 and Nginx 1.4.6 with latest HyperFastCgi source. Nginx configuration works as expected with fastcgi-mono-server4 and is the following:

server {
    listen 80; ## listen for ipv4

    location / {
                 root /home/ceco/Desktop/HelloService/;
                 index index.html index.htm default.aspx Default.aspx;
                 fastcgi_index Default.aspx;
                 fastcgi_pass unix:/tmp/hello-service;
                 include /etc/nginx/fastcgi_params;
        }
}

I start HyperFastCgi server with same arguments as fastcgi-mono-server4:

/applications=/:/home/ceco/Desktop/HelloService/ /socket=unix /filename=/tmp/hello-service

The problem is that when I make a request to my service (in the same way as with fastcgi-mono-server4) the browser is waiting for a response for about a minute and then says "Oops! Google Chrome could not connect to ...". The interesting stuff - if I close the HyperFastCgi server 1-2 seconds after the request has been sent from my browser (enough time for it to be processed) then the expected JSON is shown!

I tried to debug the code but the only thing I could notice is that NetworkConnector.ReceiveCallback is called twice:

the first time with bytesRead > 0 which I suppose is the expected request
about 5-10 seconds later again but this time bytesRead <= 0

After that the server hangs for a while and then the browser says "Oops ..." as mentioned above.

xplicit / hyperfastcgi Goto Github PK

hyperfastcgi's People

Contributors

Stargazers

Watchers

Forkers

hyperfastcgi's Issues

Recommend Projects

Recommend Topics

Recommend Org