google / libsxg Goto Github PK
View Code? Open in Web Editor NEWSigned HTTP Exchange library
License: Apache License 2.0
Signed HTTP Exchange library
License: Apache License 2.0
It would be useful if there were a CMAKE option to generate a static build of libsxg.
markdownlint found a bunch of minor issues in #55. I used the following style file:
all
exclude_rule 'MD013'
exclude_rule 'MD014'
exclude_rule 'MD024'
exclude_rule 'MD029'
There is a second project with the same name that looks similarly complete and popular:
https://github.com/DavidAnson/markdownlint
We should investigate if we can automate this linting in CI.
even when specifying CMAKE_INSTALL_PREFIX
, i get this error:
CMake Error at CMakeLists.txt:151 (install):
install TARGETS given no LIBRARY DESTINATION for shared library target
"sxg".
including a CMAKE_INSTALL_LIBDIR
fixes, so it looks like the documentation change made in 1559f01 is innacturate.
sxg_codec.c uses BIO_f_base64
for base 64 encoding, but BoringSSL lists this function as deprecated:
// Deprecated functions.
|
| // BIO_f_base64 returns a filter \|BIO\| that base64-encodes data written into
| // it, and decodes data read from it. \|BIO_gets\| is not supported. Call
| // \|BIO_flush\| when done writing, to signal that no more data are to be
| // encoded. The flag \|BIO_FLAGS_BASE64_NO_NL\| may be set to encode all the data
| // on one line.
| //
| // Use \|EVP_EncodeBlock\| and \|EVP_DecodeBase64\| instead.
| OPENSSL_EXPORT const BIO_METHOD *BIO_f_base64(void);
Any chance we could replace the calls in libsxg with EVP_EncodeBlock
/EVP_DecodeBase64
as the comments in BoringSSL suggest?
Per this comment we should update sxg_execute_ocsp_request()
in src/sxg_cert_chain.c
to add sleep/limit. This protects against:
OCSP_sendreq_nbio
that causes a hot loop that pegs the CPU running nginx.Is there any work being planned to support reading and verification of the SXG format using libsxg
?
Looks like all the callers of sxg_write_map_cbor_header
pass in a size_t
. For defense-in-depth, it would be nice to add an if (value < 0) do_something_else();
before each one is cast to a uint64_t
.
Low risk. This isn't a buffer overflow risk. Worst case is we serve an invalid SXG that the browser rejects due to CBOR parse error.
SXG generator in C++ may be convenient.
We should ensure the lib continues to work in MacOS (e.g. no regressions on the fix in #99).
The existing CI uses docker, which doesn't support MacOS as a guest OS (to my knowledge). However, GitHub Actions does, so we should add a workflow that builds libsxg directly, like #3 would've.
/cc @8W9aG
For example, gensxg uses -url
instead of -uri
and does not support -o
.
SXG is a binary format and a bit complex for normal user.
As a SXG library, this project should support parsing existing SXG files and help extracting meaningful values from them.
Modify .travis.yml to run under two setups - openssl and boringssl.
When I try to install libsxg i am getting the below error
In file included from /setups/libsxg-0.2/tests/test_util.cc:17:0:
/setups/libsxg-0.2/tests/test_util.h:17:25: fatal error: openssl/evp.h: No such file or directory
#include <openssl/evp.h>
^
compilation terminated.
my configuration command is
LDFLAGS="-L$AMPPS_PATH/lib -L/usr/local/apps/openssl-11/lib -lssl -lcrypto" CPPFLAGS="-I$AMPPS_PATH/include -I/usr/local/apps/openssl-11/include" cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/apps/libsxg -DCMAKE_INSTALL_LIBDIR=/usr/local/apps/libsxg/lib -DCMAKE_INSTALL_BINDIR=/usr/local/apps/libsxg/bin -OPENSSL_INCLUDE_DIR=/usr/local/apps/openssl-11/include -DOPENSSL_ROOT_DIR=/usr/local/apps/openssl-11
No idea how to set the path for OpenSSL, I have OpenSSL installed in custom path /usr/local/apps/openssl-11
Hello all contributors.
I'm trying to install libsxg on Centos 7 but I'm having a bit of a problem building from source. I have the following versions installed:
I refer to the install commands in this article: https://web.dev/how-to-set-up-signed-http-exchanges/ but it doesn't seem to work.
[root@centos build]# make
[ 6%] Building C object CMakeFiles/sxg.dir/src/sxg_buffer.c.o
[ 12%] Building C object CMakeFiles/sxg.dir/src/sxg_buffer_debug.c.o
[ 18%] Building C object CMakeFiles/sxg.dir/src/sxg_cbor.c.o
[ 25%] Building C object CMakeFiles/sxg.dir/src/sxg_codec.c.o
/root/libsxg/src/sxg_codec.c: In function ‘sxg_evp_sign’:
/root/libsxg/src/sxg_codec.c:247:7: warning: implicit declaration of function ‘EVP_DigestSign’ [-Wimplicit-function-declaration] (EVP_DigestSign(ctx, NULL, &sig_size, src->data, src->size) == 1) &&
[ 31%] Building C object CMakeFiles/sxg.dir/src/sxg_encoded_response.c.o
[ 37%] Building C object CMakeFiles/sxg.dir/src/sxg_generate.c.o
[ 43%] Building C object CMakeFiles/sxg.dir/src/sxg_header.c.o
[ 50%] Building C object CMakeFiles/sxg.dir/src/sxg_raw_response.c.o
[ 56%] Building C object CMakeFiles/sxg.dir/src/sxg_sig.c.o
/root/libsxg/src/sxg_sig.c: In function ‘sxg_sig_set_ed25519key’:
/root/libsxg/src/sxg_sig.c:71:3: warning: implicit declaration of function ‘EVP_PKEY_get_raw_public_key’ [-Wimplicit-function-declaration] return (EVP_PKEY_get_raw_public_key(public_key, NULL, &key_len) == 1) &&
[ 62%] Building C object CMakeFiles/sxg.dir/src/sxg_signer_list.c.o
[ 68%] Building C object CMakeFiles/sxg.dir/src/sxg_cert_chain.c.o
[ 75%] Linking C shared library libsxg.so
[ 75%] Built target sxg
[ 81%] Building C object CMakeFiles/gensxg.dir/src/gensxg.c.o
[ 87%] Linking C executable gensxg
libsxg.so.0.2: undefined reference to
EVP_DigestSign'
libsxg.so.0.2: undefined reference to EVP_PKEY_get_raw_public_key'
collect2: error: ld returned 1 exit status
make[2]: *** [gensxg] Error 1
make[1]: *** [CMakeFiles/gensxg.dir/all] Error 2
make: *** [all] Error 2
Looking forward to hearing from you. Thanks for looking at this issue.
we are seeing inconsistencies in the length of the signature, which is causing intermittent failures in CI. I was also able to reproduce in the libsxg build environment by building and running sxg_sig_test
repeatedly with this script:
attempts=0
fails=0
for i in `seq 1 500`; do
attempts=$((attempts+1));
make sxg_sig_test && ./sxg_sig_test
fails=$((fails+$?))
done
echo $attempts && echo $fails
In both environments, the signature is occasionally 4 bytes shorter than expected.
This is the output in our CI:
09:22:29 pinterest/filters/http/sxg/filter_test.cc:187: Failure
09:22:29 Expected equality of these values:
09:22:29 sxg.toString()
09:22:29 Which is: "sxg1-b3\0\0\x1Ehttps://example.org/hello.html\0\x1\x84\0\0\x84https://example.org/hello.html;cert-sha256=*unJ3rwJT2DwWlJAw1lfVLvPjeYoJh0+QUQ97zJQPZtc=*;cert-url=\"https://example.org/.well-known/amphtml/cert.cbor\";date=1610416640;expires=1611021440;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://example.org/.well-known/amphtml/validity.msg\"\xA4" "FdigestX9mi-sha256-03=0x0E2wkWVYOJ7Gq8+Kfaiyjo3gYCyaijhGGgkzjPoTo=G:statusC200Lcontent-typeItext/htmlPcontent-encodingLmi-sha256-03\0\0\0\0\0\0\x10\0<html><body>hi!</body></html>\n"
09:22:29 result
09:22:29 Which is: "sxg1-b3\0\0\x1Ehttps://example.org/hello.html\0\x1\x80\0\0\x84https://example.org/hello.html;cert-sha256=*unJ3rwJT2DwWlJAw1lfVLvPjeYoJh0+QUQ97zJQPZtc=*;cert-url=\"https://example.org/.well-known/amphtml/cert.cbor\";date=1610416640;expires=1611021440;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://example.org/.well-known/amphtml/validity.msg\"\xA4" "FdigestX9mi-sha256-03=0x0E2wkWVYOJ7Gq8+Kfaiyjo3gYCyaijhGGgkzjPoTo=G:statusC200Lcontent-typeItext/htmlPcontent-encodingLmi-sha256-03\0\0\0\0\0\0\x10\0<html><body>hi!</body></html>\n"
here is a sample of the output from building and running sxg_sig_test
repeatedly:
...
[ RUN ] SxgSig.ConstructAndRelease
[ OK ] SxgSig.ConstructAndRelease (0 ms)
[ RUN ] SxgSig.MakeSignature
/home/papazian/code/libsxg/tests/sxg_sig_test.cc:69: Failure
Expected equality of these values:
expected
Which is: "testname;cert-sha256=*WrpTHrnR9I9Cj+cizXTozEZB+BnjQKkRe8kKgme4iLU=*;cert-url=\"https://cert.test/cert.cbor\";date=0;expires=0;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://cert.test/validity.msg\""
sxg_test::BufferToString(output)
Which is: "testname;cert-sha256=*WrpTHrnR9I9Cj+cizXTozEZB+BnjQKkRe8kKgme4iLU=*;cert-url=\"https://cert.test/cert.cbor\";date=0;expires=0;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://cert.test/validity.msg\""
[ FAILED ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[ PASSED ] 1 test.
[ FAILED ] 1 test, listed below:
[ FAILED ] SxgSig.MakeSignature
1 FAILED TEST
[ 31%] Built target gtest
[ 81%] Built target sxg
[ 90%] Built target test_util
...
[ RUN ] SxgSig.MakeSignature
[ OK ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[ PASSED ] 2 tests.
[ 31%] Built target gtest
[ 81%] Built target sxg
[ 90%] Built target test_util
[100%] Built target sxg_sig_test
Running main() from /home/papazian/code/libsxg/build/third_party/gtest/googletest/src/gtest_main.cc
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from SxgSig
[ RUN ] SxgSig.ConstructAndRelease
[ OK ] SxgSig.ConstructAndRelease (0 ms)
[ RUN ] SxgSig.MakeSignature
[ OK ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[ PASSED ] 2 tests.
500
1
Is this expected? if so, can we update the libsxg tests to account for this inconsistency? that would provide a template for us to handle in our own tests.
On making this library useful from Ruby or Python or C++, sxg_buffer_t
is too closely bounded with internal implementation, we should divide it into pure pointer and length manipulation.
Presently libsxg compiles with clang but not gcc:
$ sudo make install
...
[ 50%] Building CXX object CMakeFiles/signer_fuzzer.dir/fuzzers/signer_fuzzer.cc.o
c++: error: unrecognized argument to -fsanitize= option: ‘fuzzer’
make[2]: *** [CMakeFiles/signer_fuzzer.dir/build.make:63: CMakeFiles/signer_fuzzer.dir/fuzzers/signer_fuzzer.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:337: CMakeFiles/signer_fuzzer.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
It appears that gcc does not support the fuzzer sanitizer. We should either document the clang dependency or change the Makefiles to not use the fuzzer.
sxg_add_ecdsa_signer
should return false if its name arg doesn't conform to the sh-token grammar. This would lead to an SXG parse error.
from discussion in #34 (comment)
It would be great if we see the list of build options, default values and option explanations.
Any other options?
We want to build a version of libsxg.a against BoringSSL so that we can integrate into our Envoy environment. However, BoringSSL does not provide openssl/ct.h
, which breaks sxg_cert_chain. It looks like sxg_cert_chain is only needed for the gencertchain tool (not gensxg), which means we can build a version of libsxg that can generate SXGs with BoringSSL rather than full OpenSSL.
On serving SXG, we need cbor file which contains certificates and OCSP stapling data.
As of #69 (in particular this change I requested), we get this warning when compiling against OpenSSL (not BoringSSL):
/usr/local/google/home/twifkak/devel/libsxg/src/sxg_codec.c: In function ‘sxg_base64encode_bytes’:
/usr/local/google/home/twifkak/devel/libsxg/src/sxg_codec.c:52:18: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
52 | if (out_length < length) return false;
| ^
Fix this.
Temporarily disabled in #79 because it fails to compile under boringssl. Hopefully it's an easy fix. If not, we can at least enable it only for the openssl build, by creating a new cmake option for it (default off), and overriding it in the openssl job of .travis.yml.
It might help to include the packages necessary to compile libsxg. It needs at least libssl-dev and clang but I am missing the sanitizers:
fuzzer disabled because the compiler /usr/bin/c++ does not support [-fsanitize=address,fuzzer] option
Any suggestions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.