Feature request: Add fastCGI support please about crow HOT 9 CLOSED

I personally hope you guys can prioritize this over https support since fastCGI is more important in real world.

It is also much simpler to implement fastCGI compared to https.

from crow.

Thanks for the suggestion and the information. Your main criticism seems to be against 2 things:

1. Not using a proxy
2. Using an HTTP / HTTPS proxy
   Your solution is to use FastCGI.

As far as the criticisms are concerned, We do not expect people to run Crow directly or proxy over HTTPS Even in the original repository's issues, it is mentioned that you should use a proxy and that you probably should not proxy over https.

As for using FastCGI to solve these concerns, this is where I start to disagree.

The primary comparison here is proxying over FastCGI vs HTTP/1.1

You mention that FastCGI is binary as opposed to text and supports multiplexing, which is similar to HTTP/2, and carries the same concerns that:

• Binary protocols aren't much faster than text based ones simply because the binary vs text difference only occurs in the headers, not the request / response bodies
• I'm also not sure how much the advantages of FastCGI are when the request is originally made to Nginx or Apache via HTTP, I would assume there would either be no difference or less overhead associated with http requests considering the server application wouldn't need to translate to a different protocol, although I'm not sure about that.
• The main valid part is using Unix domain sockets as opposed to TCP sockets. I've not benchmarked them, but generally it is assumed that Unix domain sockets are faster, so this would be the only valid advantage in my opinion.

Other points to consider regarding FastCGI is that:

• It requires a new set of dependencies and systems that are completely missing from Crow.
• It veers too far off the path of the original design.
• Unlike HTTP/2, I don't even know where to begin to try and understand let alone implement FastCGI.

All of that makes something like FastCGI a very had sell, let alone giving it priority over features that help complete or simplify the process of working with crow.

I like systems that run fast and efficient, but I'm not gonna lie to you. At the moment FastCGI is further down the list than HTTP/2, which I've been delaying since I started working on Crow.

There is already a PR (ipkn#379) That adds a simple form of Unix Domain Socket support to Crow. If you'd like to add FastCGI implementation yourself. I suggest you start there. I will also look into it, but once more I cannot spare much time trying to implement it, especially given how difficult it is to find people or time to work on Crow.

Thank you once more for the informative issue, and the suggestion.

from crow.

After looking into the source for a little, I think it is understandable that you didn't prioritize this since the entire structure of crow is designed around http and it might be even quite difficult to add https support to crow.

I would look into fastCGI myself and see if I can help.

from crow.

@The-EDev
Luckly, I do find a fastCGI library fastcgipp writen in C++ and it also has an example file echo.cpp.

I also checked fastCGI spec and IMHO the header is actually quite simple:

typedef struct {
    unsigned char version;
    unsigned char type;
    Big_endian_uint16_t requestId;
    Big_endian_uint16_t contentLength;
    unsigned char paddingLength;
    unsigned char reserved;
    unsigned char contentData[contentLength];
    unsigned char paddingData[paddingLength];
} FCGI_Record;

However, it does make me suspect whether fastCGI is worth it, as it is so simple and still sends information like hostname, cookies, uri via text.

It does save parsing 'name=val' by sending structure like

struct {
    nameLength; // might be 1 byte or 4 bytes
    valueLength; // same as above
    unsigned char name[];
    unsigned char value[];
}

but that doesn't make a lot of difference and might even make performance worse since it needs to also send FCGI_BeginRequestBody and FCGI_EndRequestBody.

The only benefit from fastCGI as far as I seen is the multiplexing and pretty much anything else doesn't have any real benefit for the content server.

But even multiplexing might not have any real benefits since in real world, the network between the reverse proxy and the content server are pretty good and having multiple TCP connection really isn't a big problem.

It seems that fastCGI is exactly as what it is advertised: a replacement for CGI and isn't of much benefit to crow just like what you thought @The-EDev .

from crow.

I guess the only way to make crow much faster than now is to link with nginx and act like a builtin module to it, let nginx handle all the tcp connection/http parsing and shares the same address space.

This would make crow enjoy the performance of nginx while getting mature http/http2.0 or even http3.0 for free.

from crow.

@NobodyXu I did think of that, I'm not sure how Nginx and Apache modules work, but I believe flask does something similar using Werkzeug, only thing is how do you link the 2 (Crow and the server program), but if that approach works it'll be very beneficial.

from crow.

I just took a look at one of the modules listed on nginx’s wiki: mod_zip, and found information on installation in README.markdown:

Installation

To install, compile nginx with the following option:

--add-module=/path/to/mod_zip
nginx 1.10.0 or later is required
(optional) to enable the X-Archive-Charset header, libiconv is required
http_postpone must be enabled by including at least one of the http_addition, http_slice or http_ssi modules

I also found config, which is probably important for the building of the module.

Upon further investigation of its ngx_http_zip_module.c, I was able to identify the necessary data structure for a module:

ngx_module_t  ngx_http_zip_module = {
    NGX_MODULE_V1,
    &ngx_http_zip_module_ctx,   /* module context */
    NULL,                       /* module directives */
    NGX_HTTP_MODULE,            /* module type */
    NULL,                       /* init master */
    NULL,                       /* init module */
    NULL,                       /* init process */
    NULL,                       /* init thread */
    NULL,                       /* exit thread */
    NULL,                       /* exit process */
    NULL,                       /* exit master */
    NGX_MODULE_V1_PADDING
};

from crow.

@NobodyXu I did think of that, I'm not sure how Nginx and Apache modules work, but I believe flask does something similar using Werkzeug, only thing is how do you link the 2 (Crow and the server program), but if that approach works it'll be very beneficial.

It seems that the module can either be loaded dynamically or statically as addon:

if [ $ngx_module_link = DYNAMIC ] ; then
    ngx_module_name=ngx_http_zip_module
    ngx_module_srcs="$ngx_addon_dir/ngx_http_zip_module.c $ngx_addon_dir/ngx_http_zip_parsers.c $ngx_addon_dir/ngx_http_zip_file.c $ngx_addon_dir/ngx_http_zip_headers.c"

    . auto/module
elif [ $ngx_module_link = ADDON ] ; then
    HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES ngx_http_zip_module"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_zip_module.c"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_zip_parsers.c"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_zip_file.c"
    NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_zip_headers.c"

    . auto/module    
fi

The code above is taken from config.

from crow.

The discussion here can continue at #94

from crow.