koala

人生如戏，全凭演技

Parameters

KOALA_INBOUND_ADDR: ip:port, the address inbound server will bind to, default :2514
KOALA_SUT_ADDR: ip:port, the address inbound will call, default 127.0.0.1:2515
KOALA_OUTBOUND_ADDR: ip:port, the address all outgoing traffic will be redirected to, default 127.0.0.1:2516
KOALA_LOG_FILE: STDOUT/STDERR/filepath, if using filepath, the log will rotate every hour
KOALA_LOG_LEVEL: TRACE/DEBUG/INFO/ERROR/FATAL
KOALA_INBOUND_READ_TIMEOUT: a duration string, set the timeout of inbound read response from sut
KOALA_OUTBOUND_BYPASS_PORT: port, the port of outbound will bypass, eg replay a session with xdebug and pass xdebug remote port
KOALA_GC_GLOBAL_STATUS_TIMEOUT: a duration string, set the timeout of gc for koala global status, eg thread, socket
KOALA_REPLAYING_MATCH_STRATEGY: set outbound replaying match strategy, default use chunk match strategy, support sim for similarity match
KOALA_REPLAYING_MATCH_THRESHOLD: set outbound replaying similarity match threshold
KOALA_WITCH_ADDR: ip:port, witch: a WEB UI to make log and snapshot visible, default :8318
KOALA_OUTBOUND_BYPASS_ADDR: ip:port or :port, split by comma, the address of outbound will be bypassed when recording or replaying a session, eg service discovery address

Build tags

koala_go: for go application compiled with koala-go
koala_replayer: enable replaying mode
koala_recorder: enable recording mode

koala_replayer and koala_recorder can be enabled at the same time, to benchmark recording with replaying.

Recording

intercept tcp send/recv
associate send/recv data to same thread id as "session"
request => response => request, so we can know when a "talk" (with request/response pair) is complete
use udp 127.127.127.127:127 to inform recorder with helper information.
for "system under test" using thread multiplexing (one thread doing more than one thing), map real thread id to virtual thread id by helper information.

Replaying

replaying builds on same mechanism, but much more complex

"system under test" is a process, "replayer inbound server" and "replayer outbound server" lives in same process. They are two tcp servers started by the .so loaded via LD_PRELOAD.
session to replay is injected into the process via "replayer inbound server" tcp connection
"replayer inbound server" call the "system under test" via tcp connection, store the "session id <=> inbound socket" mapping.
"system under test" call external dependencies, which is intercepted to "replayer outbound server", store the "inbound socket <=> outbound socket" mapping
"replayer outbound server" use its own socket to lookup the mapping, to find which session to replay

Gateways

koala support two modes by different gateways (https://github.com/v2pro/koala/tree/master/gateway)

libc: for application built with libc (c/c++/python/java)
go: need to compile with https://github.com/v2pro/koala-go

Real World Scenarios

Long Connection: reusing connection sequentially is not a issue, just update the mapping
Multiplexing: one thread handing multiple business processes at the same time, need "helper information"
One Way Communication: request without response, need "helper information" to cut two requests out
Greeting: protocol like mysql send greeting before request. use the ip:port or just guess, to decide if greeting is needed.

alpine系统编译不了

/go/src/github.com/didi/rdebug/koala # ./build.sh recorder
+++ dirname ./build.sh
++ cd ../.
++ pwd -P
+ RDEBUG=/go/src/github.com/didi/rdebug
+ case $1 in
+ export GOPATH=/tmp/build-golang
+ GOPATH=/tmp/build-golang
+ export 'CGO_CFLAGS=-DKOALA_LIBC_NETWORK_HOOK -DKOALA_LIBC_FILE_HOOK'
+ CGO_CFLAGS='-DKOALA_LIBC_NETWORK_HOOK -DKOALA_LIBC_FILE_HOOK'
+ export 'CGO_CPPFLAGS=-DKOALA_LIBC_NETWORK_HOOK -DKOALA_LIBC_FILE_HOOK'
+ CGO_CPPFLAGS='-DKOALA_LIBC_NETWORK_HOOK -DKOALA_LIBC_FILE_HOOK'
+ export 'CGO_CXXFLAGS=-std=c++11 -Wno-ignored-attributes'
+ CGO_CXXFLAGS='-std=c++11 -Wno-ignored-attributes'
+ exec go build -tags=koala_recorder -buildmode=c-shared -o /go/src/github.com/didi/rdebug/output/libs/koala-recorder.so github.com/didi/rdebug/koala/cmd/recorder
# github.com/didi/rdebug/koala/gateway/gw4libc
In file included from file_hook.cpp:11:
file_hook.cpp:73:11: error: redefinition of 'template<class ... Args> decltype (fopen(real::fopen::args ...)) real::fopen(Args ...)'
 INTERPOSE(fopen)(const char *filename, const char *opentype) {
           ^~~~~
interpose.h:38:10: note: in definition of macro 'INTERPOSE'
     auto NAME(Args... args) -> decltype(::NAME(args...)) { \
          ^~~~
file_hook.cpp:19:11: note: 'template<class ... Args> decltype (fopen(real::fopen::args ...)) real::fopen(Args ...)' previously declared here
 INTERPOSE(fopen64)(const char *filename, const char *opentype) {
           ^~~~~~~
interpose.h:38:10: note: in definition of macro 'INTERPOSE'
     auto NAME(Args... args) -> decltype(::NAME(args...)) { \
          ^~~~
file_hook.cpp:73:11: error: redefinition of 'FILE* fopen(const char*, const char*)'
 INTERPOSE(fopen)(const char *filename, const char *opentype) {
           ^~~~~
interpose.h:49:31: note: in definition of macro 'INTERPOSE'
   extern "C" decltype(::NAME) NAME __attribute__((weak, alias("__interpose_" #NAME))); \
                               ^~~~
file_hook.cpp:19:11: note: 'FILE* fopen(const char*, const char*)' previously defined here
 INTERPOSE(fopen64)(const char *filename, const char *opentype) {
           ^~~~~~~
interpose.h:49:31: note: in definition of macro 'INTERPOSE'
   extern "C" decltype(::NAME) NAME __attribute__((weak, alias("__interpose_" #NAME))); \
                               ^~~~
file_hook.cpp:101:11: error: redefinition of 'template<class ... Args> decltype (open(real::open::args ...)) real::open(Args ...)'
 INTERPOSE(open)(const char *filename, int flags, ...) {
           ^~~~
interpose.h:38:10: note: in definition of macro 'INTERPOSE'
     auto NAME(Args... args) -> decltype(::NAME(args...)) { \
          ^~~~
file_hook.cpp:47:11: note: 'template<class ... Args> decltype (open(real::open::args ...)) real::open(Args ...)' previously declared here
 INTERPOSE(open64)(const char *filename, int flags, mode_t mode) {
           ^~~~~~
interpose.h:38:10: note: in definition of macro 'INTERPOSE'
     auto NAME(Args... args) -> decltype(::NAME(args...)) { \
          ^~~~
file_hook.cpp:101:11: error: redefinition of 'int open(const char*, int, ...)'
 INTERPOSE(open)(const char *filename, int flags, ...) {
           ^~~~
interpose.h:49:31: note: in definition of macro 'INTERPOSE'
   extern "C" decltype(::NAME) NAME __attribute__((weak, alias("__interpose_" #NAME))); \
                               ^~~~
file_hook.cpp:47:11: note: 'int open(const char*, int, ...)' previously defined here
 INTERPOSE(open64)(const char *filename, int flags, mode_t mode) {
           ^~~~~~
interpose.h:49:31: note: in definition of macro 'INTERPOSE'
   extern "C" decltype(::NAME) NAME __attribute__((weak, alias("__interpose_" #NAME))); \
                               ^~~~

v2pro / koala Goto Github PK

koala's Introduction

koala

Parameters

Build tags

Recording

Replaying

Gateways

Real World Scenarios

koala's People

Contributors

Stargazers

Watchers

Forkers

koala's Issues

Recommend Projects

Recommend Topics

Recommend Org