stringepsilon / fbjson Goto Github PK

This could require some serious work on the parser, so I'm not sure if I can / will fix this.

The standard however allows flat values.

When there is more than one escaped quote inside a string, the second escaped quote doesn't get skipped (escaped) correctly.
Instead, it seems to be parsed as the end quote.
I'm guessing that since the next non-whitespace character isn't a comma or a curly brace, parsing fails.

HTML descriptions require style tags to be within a pair of quotes, this is how this came to my attention.

This key:value pair fails to parse, and ends parsing of any further pairs:
"description":"<p><span style=\"font-family: 'arial black', 'avant garde';\">4 Pin Test setting, with comma<\/span><\/p>"

This passes ok:
"description":"<p>4 Pin Test setting with comma, semicolon ; single quote ' and colon :<\/p>"

This, with two escaped quotes, doesn't pass:
"4PinTest","description":"<p>4 Pin Test setting with comma, semicolon ; single quote ' and colon : and&nbsp; quote \" and quote \"<\/p>"

But this, with one escaped quote, does:
"description":"<p>4 Pin Test setting with comma, semicolon ; single quote ' and colon : and&nbsp; quote \"<\/p>"

jsonitem.datatype returns -1 on fail.
jsonitem.count =5, which is the number of pairs before the failing pair, and the data is correct for those pairs.

Already a TODO in the code.

When in string, deescape \u to character.

Possible blocker: I'm currently using the normal string datatype.

It's faster for big json objects.

Hi StringEpsilon,

I actually came across this project completely randomly, and thought, "Hey, I wrote a JSON parser for FreeBASIC years ago," fbjson.

I haven't written FreeBASIC in a couple of years now, but if there's anything you want to salvage from my old library, feel free, as long as you link back to my repo (which I might archive).

Anyway, this looks cool, good job!

Problem 1 A string that ends with a single backslash gets accepted (but should be rejected)

{"key":"value\n}

Problem 2 Using an empty string without a closing double quote leads to a segfault (using command-line option -exx)

{"key":"}

http://www.freebasic.net/forum/viewtopic.php?f=8&p=219044#p219044

I'm using the covid-193.p.rapidapi.com. It returns JSON like this:

{"get":"statistics","parameters":{"country":"India"},"errors":[],"results":1,"response":[{"continent":"Asia","country":"India","population":1387975211,"cases":{"new":null,"active":164278,"critical":8944,"recovered":10448406,"1M_pop":"7757","total":10767206},"deaths":{"new":null,"1M_pop":"111","total":154522},"tests":{"1M_pop":"142475","total":197752057},"day":"2021-02-02","time":"2021-02-02T14:45:06+00:00"}]}

After parsing with JsonItem and dumping, I see this:

{
  "get" : "statistics",
  "parameters" : {
    "country" : "USA"
  },
  "errors" : ,
  "results" : 1,
  "response" : [
    {
      "continent" : "North-America",
      "country" : "USA",
      "population" : 332145637,
      "cases" : {
        "new" : "+1597",
        "active" : 9825868,
        "critical" : 23860,
        "recovered" : 16632858,
        "1M_pop" : "81028",
        "total" : 
      },
      "deaths" : {
        "new" : "+33",
        "1M_pop" : "1368",
        "total" : 
      },
      "tests" : {
        "1M_pop" : "945964",
        "total" : 
      },
      "day" : "2021-02-02",
      "time" : "2021-02-02T14:45:06+00:00"
    }
  ]
}

Numbers directly followed by ,, such as "recovered":10448406, are parsed OK. However, "total":154522} isn't parsed -- with an empty string being captured.

The JSON spec does not accept \uXXXXX, instead a UTF-16 surrogate pair must be used. Right now, the code doesn't check if an escape sequence is part of a surrogate pair and tries to convert each surrogate to UTF-8. This results in invalid UTF-8 codepoints.

This means that some characters, like certain emojis, can't be properly read from their escaped notation as of now.

Quote from AGS:

The parser rejects a json file that contains a floating point number starting with a sign. Parsing the following input sets the flag _isMalformed to true (indicating parsing failed).

{"key":+4.44}

http://www.freebasic.net/forum/viewtopic.php?f=8&p=219044#p219044

Title.

Basically: Keep track if it's still okay to reference the original string and if the "copy" can / should be deallocated normally.

I am not sure how viable this actually is, but it should be worth trying out.