Proposal: more abstracted SAX handler for lots of JSON manipulates

Sonic-Cpp

A fast JSON serializing & deserializing library, accelerated by SIMD.

Requirement

c++11 or above
x86 platform with AVX2 instruction
GCC or LLVM compiler (not support MSVC compiler now)
Linux OS

Features

Complete APIs for JSON value manipulation
Fast on JSON serializing and parsing
Support parse ondemand

Benchmarks

use CMake

cmake -S . -B build -DBUILD_BENCH=ON
cmake --build build --target bench -j
./build/benchmark/bench

use bazel

bazel run :benchmark --compilation_mode=opt

Performance by sonic benchmark

Run

# build by bazel
python3 ./scripts/tools/draw-decode-encode.py

Result

Parsing Performance

Serializing performance

Performance by third-party benchmark

Below data is test by https://github.com/miloyip/nativejson-benchmark:

Parsing Performance

Serializing Performance

API Document

Make sure Doxygen 1.8.13 or higher version has been installed. Then following:

mdkir api-doc && cd api-doc
doxygen ../Doxyfile

Design

Sonic-cpp parses JSON into a compact document tree. The document structure is as follows:

There are many optimizations in parsing as follows:

using SIMD to accelerate skipping white space.
using SIMD to find escaped chars when parsing strings.
using the STOA float pointing algorithm.

Sonic-cpp serializes a document to JSON. When serializing JSON strings, we should check the escaped characters first. So, we use SIMD instructions(AVX2/SSE) to find the escaped char for a long JSON string.

Sonic-cpp also supports ParseOnDemand if the user knows the target key at compile time. ParseOndemand also used SIMD and bit manipulation to skip the unwanted values fastly.

Usage

Include

Sonic-Cpp is a header-only library, you only need to include the directory of Sonic-Cpp header files, such as adding -I/path/to/sonic/include/ to your compiler.

Parsing and Serializing

#include "sonic/sonic.h"

#include <string>
#include <iostream>

int main()
{
  std::string json = R"(
    {
      "a": 1,
      "b": 2
    }
  )";

  sonic_json::Document doc;
  doc.Parse(json);

  sonic_json::WriteBuffer wb;
  doc.Serialize(wb);
  std::cout << wb.ToString() << std::endl;
}
// g++ -I./include/ -march=haswell --std=c++11 -O3 example/parse_and_serialize.cpp -o example/parse_and_serialize

Checking parse result

#include "sonic/sonic.h"

#include <string>
#include <iostream>

int main()
{
  std::string json = R"(
    {
      "a": 1,
      "b": 2
    }
  )";

  sonic_json::Document doc;
  doc.Parse(json);
  if (doc.HasParseError()) {
    std::cout << "Parse failed!\n";
  } else {
    std::cout << "Parse successful!\n";
  }
  return 0;
}
// g++ -I./include/ -march=haswell --std=c++11 -O3 example/check_parse_result.cpp -o example/check_parse_result

Getting and Setting

#include "sonic/sonic.h"

#include <string>
#include <iostream>

using member_itr_type = typename sonic_json::Document::MemberIterator;

void print_member(member_itr_type m) {
  const sonic_json::Node& key = m->name;
  sonic_json::Node& value = m->value;
  if (key.IsString()) {
    std::cout << "Key is: "
              << key.GetString()
              << std::endl;
  } else {
    std::cout << "Incorrect key type!\n";
    return;
  }
  if (value.IsInt64()) {
    std::cout << "Value is " << value.GetInt64() << std::endl;
  }

  return;
}

void set_new_value(member_itr_type m) {
  sonic_json::Node& value = m->value;
  value.SetInt64(2);
  return;
}

int main()
{
  std::string json = R"(
    {
      "a": 1,
      "b": 2
    }
  )";

  sonic_json::Document doc;
  doc.Parse(json);

  if (doc.HasParseError()) {
    std::cout << "Parse failed!\n";
    return -1;
  }

  // Find member by key
  if (!doc.IsObject()) { // Check JSON value type.
    std::cout << "Incorrect doc type!\n";
    return -1;
  }
  auto m = doc.FindMember("a");
  if (m != doc.MemberEnd()) {
    std::cout << "Before Setting new value:\n";
    print_member(m);
    std::cout << "After Setting value:\n";
    set_new_value(m);
    print_member(m);
  } else {
    std::cout << "Find key doesn't exist!\n";
  }
  return 0;
}
// g++ -I./include/ -march=haswell --std=c++11 -O3 example/get_and_set.cpp -o example/get_and_set

The following Is*, Get* and Set* methods are supported:

IsNull(), SetNull()
IsBool(), GetBool(), SetBool(bool)
IsString(), GetString(), GetStringView(), SetString(const char*, size_t)
IsNumber()
IsArray(), SetArray()
IsObject(), SetObject()
IsTrue(), IsFalse()
IsDouble(), GetDouble(), SetDouble(double)
IsInt64(), GetInt64(), SetInt64(int64_t)
IsUint64(), GetUint64(), SetUint64_t(uint64_t)

More usage.

RoadMap

Contributing

Please read CONTRIBUTING.md for information on contributing to sonic-cpp.

	### Find member in object
	There are two ways to find members: `operator[]` or `FindMember`. We recommand
	using `FindMember`.
	```c++
	#include "sonic/sonic.h"
	// ...
	using AllocatorType = typename sonic_json::Allocator;
	sonic_json::Node node;
	AllocatorType alloc;
	// Add members for node

	// find member by key
	if (node.IsObject()) { // Note: CHECK NODE TYPE IS VERY IMPORTANT.
	const char* key = "key";
	auto m = node.FindMember(key, std::strlen(key)); // recommand
	if (m != node.MemberEnd()) {
	// do something
	}

	auto m = node.FindMember(key, std::strlen(key), alloc); // create a map to record k-v.
	if (m != node.MemberEnd()) {
	// do something
	}

	{
	sonic_json::Node& value = node[key];
	// do something
	}
	}
	```

	A multimap will be created to acclerate find operation if `FindMember` has allocator
	as argument. This map is maintained by object, user cannot access it directly.

bytedance / sonic-cpp Goto Github PK

sonic-cpp's Introduction

Sonic-Cpp

Requirement

Features

Benchmarks

Performance by sonic benchmark

Performance by third-party benchmark

API Document

Design

Usage

Include

Parsing and Serializing

Checking parse result

Getting and Setting

RoadMap

Contributing

sonic-cpp's People

Contributors

Stargazers

Watchers

Forkers

sonic-cpp's Issues

What happened

Benchmark

Optimize:

Enhancements

Recommend Projects

Recommend Topics

Recommend Org