yhirose / cpp-httplib Goto Github PK
View Code? Open in Web Editor NEWA C++ header-only HTTP/HTTPS server and client library
License: MIT License
A C++ header-only HTTP/HTTPS server and client library
License: MIT License
Hey there,
I really like the idea behind this library, but I am running into a few issues when trying to include this with my project. I am using g++ with Mingw on Windows, rather than Visual Studio.
My current command to run everything looks like this:
g++ test.cpp httplib.h -DWEBVIEW_WINAPI=1 -lole32 -lcomctl32 -loleaut32 -luuid -mwindows -DGTEST_USE_OWN_TR1_TUPLE -I.. -I. -Wall -Wextraw
I am also using a Webview library to create a browser. The errors that I receive are related to " undefined reference to `_imp__freeaddrinfo@4'" for example, so linker errors.
Any ideas?
POST requests are extremely slow. Each request is taking about 1 second to complete. I am using the library on windows.
There are no 'Put' in the current example of 'Put'. :-) (README)
DJuego
See #71 for the detail.
Add static files support. Need a method like 'set_htdocs_path'.
I need to build a static executable using cpp-httplib.
Then I add -static
to CFLAGS
and -ldl -Wl,--whole-archive -lpthread -Wl,--no-whole-archive
to the LDFLAGS
.
After my changes the test/Makefile
contains:
CFLAGS = -std=c++11 -DGTEST_USE_OWN_TR1_TUPLE -I.. -I. -Wall -Wextra -static
OPENSSL_SUPPORT = -DCPPHTTPLIB_OPENSSL_SUPPORT -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib -lssl -lcrypto
ZLIB_SUPPORT = -DCPPHTTPLIB_ZLIB_SUPPORT -lz
EXTRA_LDFLAGS = -Wl,--whole-archive -lpthread -Wl,--no-whole-archive -ldl
all : test
./test
test : test.cc ../httplib.h Makefile
$(CC) -o test $(CFLAGS) test.cc gtest/gtest-all.cc gtest/gtest_main.cc $(OPENSSL_SUPPORT) $(ZLIB_SUPPORT) $(EXTRA_LDFLAGS)
pem:
openssl genrsa 2048 > key.pem
openssl req -new -key key.pem | openssl x509 -days 3650 -req -signkey key.pem > cert.pem
clean:
rm test *.pem
When I run the test suite its ok until
[----------] 3 tests from ConnectionErrorTest
[ RUN ] ConnectionErrorTest.InvalidHost
Makefile:9: recipe for target 'all' failed
make: *** [all] Falha de segmentação
Need an easy way to access it, perhaps in request structure
Under POST with parameters in the README it is showed how to use httplib::Map
to post data. This does not exist anymore and should be updated to httplib::Params
(?).
It would be nice to have optional support for gzip.
It would be really helpfull to split the library into multiple .hpp
files for development and then merge them into single .hpp
/ .h
for distribution :)
Examples:
https://github.com/ThePhD/sol2/blob/develop/single.py
https://github.com/oftc/jsoncpp/blob/master/amalgamate.py
https://github.com/connormanning/arbiter/blob/master/amalgamate.py
This would not only allow for easier development but also for possible PRs/contributions or simple modifications/extensions that others might need to make.
It would also improve code reuse and abstraction mechanisms.
For example sprintf_socket
could be made as a method writefmt
of Stream
and not implementation detail.
when writing the response body to a file, i found that it contained extra unwanted data. I was trying to make it get a file from a api, the data is in base64 and it appends value before the expected data
the expected result =
Any help would be appreciated
used code -
httplib::SSLClient client("example.com", 443);
auto logincheck = "/load?username=" + user + "&product=" + product;
const char* chr = _strdup(logincheck.c_str());
auto res = client.get(chr);
if (res && res->status == 200)
encoded << res->body;
sorry for the bad code format
edit : found that it might be caused by it displaying chunks as the server is sending it with "Transfer-Encoding: chunked"
Currently you are assuming that windows compiler is always MSVC (by checking_MSC_VER) which might not be true (as is in my case).
You should change it to check for (_WIN32) in all places first.
Also you might consider adding autodetection for headers when c++17 compiler is found...
Example:
#if ( __cplusplus > 201703L)
#if __has_include(<openssl/ssl.h>)
#ifndef CPPHTTPLIB_OPENSSL_SUPPORT
#define CPPHTTPLIB_OPENSSL_SUPPORT
#endif
#endif
#endif
Thank you for this project! Thank you for your effort!
Header-only, no dependencies. cross-platform, light and elegant... It's the closest thing to the solution I was looking for.
Only I miss a more complete interface (with PUT and DELETE and perhaps, others). Any reason not to implement those features?
In any case I am not an expert and I think that what your project offers is enough for my purposes. :-)
Thanks again!
DJuego
Need solution file for VC11.
The client, when doing a request to a server, does not include the host in the header. This causes some failures on some sites. Something like this fixes the problem (presumably also needed for HEAD and POST methods):
@@ -1078,6 +1084,7 @@ inline std::shared_ptr<Response> Client::get(const char* path)
Request req;
req.method = "GET";
req.path = path;
+ req.set_header("Host", host_.c_str());
auto res = std::make_shared<Response>();
First of all, great little library!
Is it possible to get the full URL as sent by the client from the Request struct? I know I can get the path and params, but sometimes it's useful to get the full URL.
So, if the browser requests: http://127.0.0.1:1234/app?var1=value1&var2=value2
With Request::path I can get e . g. /app. With Request::params I can access the variables, but seems there is no way to access the full URL (besides reconstructing it yourself)?
I think something like Request::url which would return e. g. http://127.0.0.1:1234/app?var1=value1&var2=value2 or at least app?var1=value1&var2=value2 would be quite useful.
Multi-thread support to accept simultaneous connections as bellow.
ab -n 10 -c 5 http://host:port/slow_request_which_takes_10_secondes
Allow users to disable this feature with CPPHTTPLIB_NO_MULTI_THREAD_SUPPORT
.
It would be really nice if we could process forms encoded multipart/form-data, including file uploads. Unfortunately this feature lacks in most lightweight C++ HTTP libraries, though it doesn't takes much slocs to implement.
Having a pretty simple server, which is set up very close to the examples.
After some time of fine running, it just starts failing with SIGPIPE here:
inline int SocketStream::write(const char* ptr, size_t size) // ptr = "Connection: close\r\n"
{
return send(sock_, ptr, size, 0);
}
Linux 4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 x86_64 x86_64
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)
Update:
Sometimes it runs really long without it. So one can say, that sometimes it is not happening.
I was trying to use cpp-httplib for running tests, so I'd like to start it on a system-selected port, and then have clients connect to that port. I don't want to pre-select a port number to avoid collisions with other things on the system, or when tests are run in parallel.
When I do the common thing of passing 0 as the server port everything seems to work, and I was able to hunt down the actual port by running netstat, and when I then connect to that port things work just fine.
Would it be possible to have a method to retrieve port that the server's running on? (Or is that already possible and I missed how to do so?)
If the HTTP server the client is requesting from sends a large page, the client will not read all of the response, because there is only one call made to the socket recv() method.
Something like this will fix it:
@@ -508,10 +508,16 @@ bool read_content(Stream& strm, T& x)
{
auto len = get_header_value_int(x.headers, "Content-Length", 0);
if (len) {
x.body.assign(len, 0);
- if (!strm.read(&x.body[0], x.body.size())) {
- return false;
- }
+ auto r = 0;
+ while(r < len){
+ auto r_incr = strm.read(&x.body[r], x.body.size());
+ r += r_incr;
+ if(r_incr <= 0) break;
+ }
+ if (!r) {
+ return false;
+ }
}
return true;
}
I'm finding I have crashes on shutdown due to https://github.com/yhirose/cpp-httplib/blob/master/httplib.h#L1596.
In this case, the worker thread is copying this
by value. If, while the thread is servicing the request the server is shutdown, listen() returns, and Server is destroyed, then e.g. Server::process_request
can try to access freed memory running on the thread.
A simple way to fix this would be to have listen()
store the std::threads instead of detach()
ing them and then .join()
before exiting listen()
. However, that would be very inefficient for a long running server as that list would grow during each request and not be cleared until stop()
.
Implementing the "// TODO: Use thread pool..." near there might be a good way to fix this (assuming the pool is joined before shutdown).
A more direct slightly hacky way might be to increment an atomic when a thread is started, decrement it on thread exit, and wait until it's zero before exiting listen()
.
Allow server to accept case insensitive HTTP header such as 'content-type: text/plain'.
It is allowed in HTTP 1.0 to have no "Content-Length" header field in the response-header: In this case closing of the connection indicates the end of the response-body. But in the current implementation reading the body of the response is skipped if there is no "Content-Length" field, resulting into an empty body.
The template function read_content(Stream& strm, T& x)
is used for the server (handle request) and for the client (handle response), but the handling for no "Content-Length" should only be applied to the response.
An example which demonstrates the problem (openssl-1.0.2l, Windows, Visual Studio 2017):
#include <iostream>
#define CPPHTTPLIB_OPENSSL_SUPPORT
#include <httplib.h>
#pragma comment(lib, "Ws2_32.lib")
#pragma comment(lib, "libeay32.lib")
#pragma comment(lib, "ssleay32.lib")
int main(int argc, char* argv[]) {
httplib::SSLClient cli("bittrex.com", 443);
auto res = cli.get("/api/v1.1/public/getmarkets");
if (res && res->status == 200) {
std::cout << res->body << std::endl;
}
return 0;
}
I have created a template specialization for Response
which will work (but is maybe not optimal, because it adds bytes to the body one-by-one, similar to socket_gets
).
bool read_content(Stream& strm, Response& x)
{
auto len = get_header_value_int(x.headers, "Content-Length", 0);
if (len) {
x.body.assign(len, 0);
auto r = 0;
while (r < len) {
auto r_incr = strm.read(&x.body[r], len - r);
if (r_incr <= 0) {
return false;
}
r += r_incr;
}
} else { /* Start Added code */
for (;;) {
char byte;
auto n = strm.read(&byte, 1);
if (n < 1) {
if (x.body.size() == 0) {
return true; // no body
}
else {
break;
}
}
x.body += byte;
}
} /* End Added code */
return true;
}
This could cause bad security problem.
I think this only really affects the test code, but I thought perhaps you would want to know.
In investigating a possible patch for #46, I split listen()
into helper functions locally. After I did that I was experiencing an occasional crash in the server tests. I made a repro by adding a short sleep() in the body of listen() on master like this: https://github.com/yhirose/cpp-httplib/compare/master...sgraham:start-stop-crash?expand=1
When I run this (on a Mac), ASAN reports a crash in the thread:
https://gist.github.com/sgraham/461f286144ef8c7e531dad75ec2960d1
I think this is because the svr_.is_running()
in the test SetUp() passes while listen()
is in the sleep, then the server is immediately stopped, then the thread continues on to detail::select_read()
, but svr_sock_
is now -1, so select_read()
crashes in FD_SET(-1, &fds)
.
(Sorry for the long explanation without a fix, I wasn't immediately sure where the correct place to synchronize would be.)
When using the code sample :
` ``
std::thread signaling_health_thread([] () {
printf("hey from new thread!\n");
httplib::Client cli("localhost", 1234);
auto res = cli.get("/hi");
printf("I have got a res!\n");
if (res && res->status == 200) {
std::cout << res->body << std::endl;
}
});
signaling_health_thread.join();
I get a SIGPIPE signal if there is no port open on localhost:1234.
Is that expected ? Thanks.
First of all, thanks for providing such a great library!
I had a problem to get connections to certain HTTPS hosts working with using httplib::SSLClient
. After investigating with openssl s_client, I found that the problem is that the host sends the wrong certificate to the client. This is because, the server doesn't know for which site the client expects the certificate if multiple sites/certificates are on the same host (i.e. on a virtual server).
The solution to this problem in TLS is the extension RFC 6066, Server Name Indication. Using this TLS extension the client will send during the handshake the servername for which he wants to receive the certificate, so that the correct certificate can be supplied to the client. The extension is already implemented in openssl and can be used with the "-servername" parameter, i.e. openssl s_client -connect httpbin.org:443 -servername httpbin.org
SNI (Server Name Indication) is widely used on the internet and I think it would be a valuable addition to cpp-httplib.
A minimal example which shows the problem using httplib::SSLClient (openssl-1.0.2l, Visual Studio 2017):
#include <iostream>
#define CPPHTTPLIB_OPENSSL_SUPPORT
#include <httplib.h>
#pragma comment(lib, "Ws2_32.lib")
#pragma comment(lib, "libeay32.lib")
#pragma comment(lib, "ssleay32.lib")
int main(int argc, char* argv[]) {
httplib::SSLClient cli("httpbin.org", 443);
auto res = cli.get("/get");
if (res && res->status == 200) {
std::cout << res->body << std::endl;
}
return 0;
}
One possible workaround in read_and_close_socket_ssl
is:
SSL_set_bio(ssl, bio, bio);
SSL_set_tlsext_host_name(ssl, "httpbin.org"); // workaround
SSL_connect_or_accept(ssl);
It would be more elegant, if the hostname would come from the Client::host_
, but it is not accessible from there with the current interface.
In RFC 7230(HTTP1.1),
if the connection's incoming TCP port number
differs from the default port for the effective request URI's
scheme, then a colon (":") and the incoming port number (in
decimal form) are appended to the authority component.
While I know your library supports HTTP/1.0,
I'm glad to append port number to HOST header like following:
#- req.set_header("Host", host_.c_str());
+ req.set_header("Host", (host_ + ":" + std::to_string(port_)).c_str());
added functionality to get downloading portions of big files using range_low & range_high
can be easily achieved with following code:
( default parameter values must be defined in the class definition, they are just in the implementation for ovesight )
`
inline void Client::add_range_header (Request &req, int64_t range_low, int64_t range_high)
{
std::string range_str = "bytes=";
if ( range_low >= 0 )
{
range_str += std::to_string (range_low);
}
range_str += "-";
if ( range_high >= 0 )
range_str += std::to_string (range_high);
req.set_header ("Range", range_str.c_str ());
}
inline std::shared_ptr<Response> Client::get (const char* path, Progress callback, int64_t range_low=-1, int64_t range_high=-1)
{
Request req;
req.method = "GET";
req.path = path;
req.progress = callback;
add_default_headers (req);
if ( range_low >= 0 )
add_range_header (req, range_low, range_high);
auto res = std::make_shared<Response> ();
return send (req, *res) ? res : nullptr;
}
`
It would be nice if Client:send
added default_headers (HOST
, User-Agent
, Accept
) when they are not set :)
Hi again!
I try cpp-httplib with MinGW (x86_64-7.3.0-release-win32-seh-rt_v5-rev0.7z)
I have had to add to httplib.h the lines (and #define WIN32_ and #define MINGW)
#ifdef MINGW
#include "mingw.mutex.h"
#include "mingw.thread.h"
#else
#include <mutex>
#include <thread>
#endif
NOTE: I got mingw.mutex.h and mingw.thread.h from https://github.com/meganz/mingw-std-threads
After that, i get:
..\..\inc\cpp-httplib/httplib.h: In function 'std::__cxx11::string httplib::detail::get_remote_addr(socket_t)':
..\..\inc\cpp-httplib/httplib.h:629:9: error: 'inet_ntop' was not declared in this scope
inet_ntop(AF_INET, &s->sin_addr, ipstr, sizeof(ipstr));
^~~~~~~~~
..\..\inc\cpp-httplib/httplib.h:629:9: note: suggested alternative: 'inet_ntoa'
inet_ntop(AF_INET, &s->sin_addr, ipstr, sizeof(ipstr));
^~~~~~~~~
inet_ntoa
..\..\inc\cpp-httplib/httplib.h:633:9: error: 'inet_ntop' was not declared in this scope
inet_ntop(AF_INET6, &s->sin6_addr, ipstr, sizeof(ipstr));
^~~~~~~~~
..\..\inc\cpp-httplib/httplib.h:633:9: note: suggested alternative: 'inet_ntoa'
inet_ntop(AF_INET6, &s->sin6_addr, ipstr, sizeof(ipstr));
^~~~~~~~~
inet_ntoa
Apparently, I have solved this problem by adding the following code to httplib.h.
#ifdef MINGW
const char* inet_ntop(int af, const void* src, char* dst, int cnt){
struct sockaddr_in srcaddr;
memset(&srcaddr, 0, sizeof(struct sockaddr_in));
memcpy(&(srcaddr.sin_addr), src, sizeof(srcaddr.sin_addr));
srcaddr.sin_family = af;
if (WSAAddressToString((struct sockaddr*) &srcaddr, sizeof(struct sockaddr_in), 0, dst, (LPDWORD) &cnt) != 0) {
DWORD rv = WSAGetLastError();
printf("WSAAddressToString() : %d\n",rv);
return NULL;
}
return dst;
}
#endif
I am going to test it but...Is there any better solution?
DJuego
Hello,
I had been using the default setting for the server which was HTTP/1.0 until recently. In test situations, the client code sends one request, and after the server has sent its response, it calls Server::stop(), ending the test. This still works correctly, however, I believe because of the change to 1.1, causes the server to wait for 5s (CPPHTTPLIB_KEEPALIVE_TIMEOUT_SECOND) before termination, which makes the tests take longer than necessary.
I can work around this by locally patching CPPHTTPLIB_KEEPALIVE_MAX_COUNT to 1, but I was just wondering if there's something different I should be sending from the client (I'm only using cpp-httplib for the server, not the client side). Alternatively, maybe it would be possible to have Server::stop() signal to the select() somehow that it should shut down immediately?
How can we define custom timeouts for requests?
My current code:
httplib::Client cli(serverAddress.c_str(), port);
auto res = cli.post(path, data, "text/plain");
if (res == nullptr) {
LOGE() << "failed to send data to " << path << ": got no response";
return;
}
if (res->status != 200) {
LOGE() << "failed to send data to " << path << ", response code: " << res->status;
return;
}
This just blocks when the server is not reachable. I'd like to report an error after 5 seconds.
First of all. Contratulation on your excellent work. Iam coming from nodejs world so i found it easy to understand your vision.
Does this library support some kind of middleware functions (auth or ip filter middleware etc.).
Secondly, is there any benchmark that compares performance with other frameworks ? (cpprestapi
casablanca etc.)
Thank you.
Hi,
I just starting trying your library and noticed a warning:
cpp-httplib/httplib.h:256:54: warning: unused parameter ‘bufsiz’ [-Wunused-parameter]
inline bool socket_gets(Stream& strm, char* buf, int bufsiz)
I could ignore it, but since it's a buffer size, wouldn't that be a bug and the bufsize should be used to prevent buffer overflow, no ?
Thanks
Baptiste
How does the library Server object support stopping the listener ?
how to post file to server, content_type multpart/form-data
Some browsers seem to send metadata about MultipartFiles as lower-case 'content-type' and 'content-disposition'. This seems to be consistent with the specification (RFC7578). Currently, these are missed with this http server and are therefore not handled correctly.
The simple diff below changes the regex(es) that look for this metadata to case-insensitive and fixes this problem.
$ git diff httplib.h
diff --git a/httplib.h b/httplib.h
index 1716d0e..461c1e4 100644
--- a/httplib.h
+++ b/httplib.h
@@ -1026,10 +1026,9 @@ inline bool parse_multipart_formdata(
static std::string crlf = "\r\n";
static std::regex re_content_type(
- "Content-Type: (.*?)");
+ "Content-Type: (.*?)", std::regex_constants::icase);
static std::regex re_content_disposition(
- "Content-Disposition: form-data; name=\"(.*?)\"(?:; filename=\"(.*?)\")?");
+ "Content-Disposition: form-data; name=\"(.*?)\"(?:; filename=\"(.*?)\")?", std::regex_constants::icase);
auto dash_boundary = dash + boundary;
Every now and then my server crashes with following error.
main: ext/httplib.h:651: std::string httplib::detail::decode_url(const string&): Assertion `s[i]' failed.
The first file is certificate, the second is private key, not the other way around.
LNK2019 unresolved external symbol __imp_WSAStartup referenced in function "public: __cdecl httplib::detail::WSInit::WSInit(void)" (??0WSInit@detail@httplib@@qeaa@XZ)
got this error when i add <httplib.h> file in my project....
pls help...
OpenSSL support.
Hello
Thanks for the lib!
I got an issue with "httplib::Params".
According to readme we can fill in post parameters like this :
httplib::Params params;
params["name"] = "john";
But it doesn't compile : error C2676: binary '[': 'httplib::Params' does not define this operator or a conversion to a type acceptable to the predefined operator
Params is a std::multimap and it seems it doesn't support this way to insert element.
Hi!
It is said that cpp-httplib is C++11 library. But when I try to build example on gcc, I get an error:
error: use of ‘auto’ in lambda parameter declaration only available with -std=c++14 or -std=gnu++14
While this relates only to examples code it... should be at least said somewhere...
Great library -- one issue is that the sample code in the README.md doesn't compile because
the parameters const Response& res
are actually supposed to be non-const: Response& res
svr.get("/hi", [](const Request& req, const Response& res) {
res.set_content("Hello World!", "text/plain");
});
and
svr.get(R"(/numbers/(\d+))", [&](const Request& req, const Response& res) {
auto numbers = req.matches[1];
res.set_content(numbers, "text/plain");
});
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.