go-faster / jx Goto Github PK

Ref: romshark/jscan#1

Currently it can allocate(?) in slow path.

Provide fast decoder for UUIDs, time.Time, netip.IP, etc. May be useful for ogen purposes.

See encoding/decoding optimization for UUID ogen-go/ogen#98.

Currently there is nothing on https://pkg.go.dev/github.com/ogen-go/jx

Request “github.com/ogen-go/jx” does nothing except showing error.

Ref: golang/go#49237

• Use Go test naming convention (TestDecoder_Skip instead of Test_skip)
• Unify benchmarks and delete duplicates (BenchmarkValid vs BenchmarkSkip)
• Populate testdata with different cases (real world objects, big array of primitives, use some benchmark sets)
  • #12 (added floats.json)
  • #15 (added small/medium/large/etc)
  • #18 (added bools.json and nulls.json)
  • e00d36c (added corpus from JSONTestSuite)
• Use table tests, generate cases for in-memory and streaming decoding

Currently, float decoder fallbacks to strconv.ParseFloat and causes []byte -> string allocation. Possibly we could avoid such conversion by vendoring float implementation from Go library. Also we should try to improve current float parsing algorithm.

i have a use case where i would like part of my message to be json streamed encoded from reflection, but i would still like to use jx for the remaining encoding, as most requests are small and do not require any reflection.

currently, when the writer is in streaming mode, all calls to Write will cause an error.

i propose to simply flush the buffer and then pass the call to Write, which would allow now other json-encoders to stream into jx's encoder. i did this in a fork https://github.com/elee1766/jx and it seems to be working?

given that the package does not want to do any reflection based encoding, this seems like a good compromise?

Currently we use some bytesets to improve matching speed

• To match space characters
  

  jx/dec_read.go

  Lines 15 to 17 in 3ed0c1e

  	var spaceSet = [256]byte{
  	' ': 1, '\n': 1, '\t': 1, '\r': 1,
  	}

• To match digits and skip number
  

  jx/dec_skip.go

  Lines 47 to 66 in 97d4e11

  	skipNumberSet = [256]byte{
  	'0': 1,
  	'1': 1,
  	'2': 1,
  	'3': 1,
  	'4': 1,
  	'5': 1,
  	'6': 1,
  	'7': 1,
  	'8': 1,
  	'9': 1,
  	',': 2,
  	']': 2,
  	'}': 2,
  	' ': 2,
  	'\t': 2,
  	'\n': 2,
  	'\r': 2,
  	}

• To match and validate hex and fast string escaping
  

  jx/dec_skip.go

  Lines 242 to 256 in 97d4e11

  	escapedStrSet = [256]byte{
  	'"': 1, '\\': 1, '/': 1, 'b': 1, 'f': 1, 'n': 1, 'r': 1, 't': 1,
  	'u': 2,
  	}
  	hexSet = [256]byte{
  	'0': 1, '1': 1, '2': 1, '3': 1,
  	'4': 1, '5': 1, '6': 1, '7': 1,
  	'8': 1, '9': 1,
  	'A': 1, 'B': 1, 'C': 1, 'D': 1,
  	'E': 1, 'F': 1,
  	'a': 1, 'b': 1, 'c': 1, 'd': 1,
  	'e': 1, 'f': 1,
  	}

  
  

  jx/w_str.go

  Lines 211 to 219 in c048666

  	var safeSet = [256]byte{
  	// First 31 characters.
  	1, 1, 1, 1, 1, 1, 1, 1,
  	1, 1, 1, 1, 1, 1, 1, 1,
  	1, 1, 1, 1, 1, 1, 1, 1,
  	1, 1, 1, 1, 1, 1, 1, 1,
  	'"': 1,
  	'\\': 1,
  	}

• To make HTML-safe escaping
  

  jx/w_str.go

  Line 14 in c048666

  var htmlSafeSet = [utf8.RuneSelf]bool{

Totaly we have 6 sets, 5 of [256]byte and one [128]byte, it tooks 256*5 + 128 = 1408 bytes.
In benchmarks it performs well because we use only 1-2 of them.
But it may cause cache pollution and slow down parser in realistic cases.

We need to research real impact of such optimizations and possibly use comparsion instead or merge some sets and use bit masks.

Like #33, but for objects.