Manipulate JSON in an object-oriented way and even do four operations JSON library

foreword

In the previously implemented JSON parser , only a JSON string was converted into a JSONObject , and it was not mapped to a specific struct ; If you want to get the value, you need to make an assertion to convert it to map or slice it and then get it, it will be more troublesome.

 decode, err := gjson.Decode(`{"glossary":{"title":"example glossary","age":1}}`)
    assert.Nil(t, err)
    glossary := v["glossary"].(map[string]interface{})
    assert.Equal(t, glossary["title"], "example glossary")
    assert.Equal(t, glossary["age"], 1)

But in fact, after thinking about it, in some scenarios, we only need to get the value of a field in JSON , so we need to declare a struct which will be a little troublesome.

After inquiries, I found that there is already a similar library to solve this problem, https://github.com/tidwall/gjson and there are still many stars (even the names are the same 😂), indicating that this kind of demand is still very strong. .

So I also plan to add similar functions, which are used as follows:

Finally, a four arithmetic function is added.

Object-oriented way to manipulate JSON

Because the functions are similar, I refer to the tidwall of API , but remove some features that I think are temporarily unavailable, and adjust the syntax a little.

The current version can only access the data through the determined key plus . dot notation, if it is an array, use [index] to access the subscript.
[] Symbolic access to arrays I think is more intuitive.

Here is an example of an access with multiple nesting JSON :

 str := `
{
"name": "bob",
"age": 20,
"skill": {
    "lang": [
        {
            "go": {
                "feature": [
                    "goroutine",
                    "channel",
                    "simple",
                    true
                ]
            }
        }
    ]
}
}`

name := gjson.Get(str, "name")
assert.Equal(t, name.String(), "bob")

age := gjson.Get(str, "age")
assert.Equal(t, age.Int(), 20)

assert.Equal(t, gjson.Get(str,"skill.lang[0].go.feature[0]").String(), "goroutine")
assert.Equal(t, gjson.Get(str,"skill.lang[0].go.feature[1]").String(), "channel")
assert.Equal(t, gjson.Get(str,"skill.lang[0].go.feature[2]").String(), "simple")
assert.Equal(t, gjson.Get(str,"skill.lang[0].go.feature[3]").Bool(), true)

Personally, I feel that the use of such a grammar is quite intuitive, and I believe it is relatively simple for users.

The return value refers to tidwall uses a Result object, which provides a variety of methods to easily obtain various types of data

 func (r Result) String() string
func (r Result) Bool() bool
func (r Result) Int() int
func (r Result) Float() float64
func (r Result) Map() map[string]interface{}
func (r Result) Array() *[]interface{}
func (r Result) Exists() bool

For example, using Map()/Array() these two functions can map JSON data to map and slice, of course, provided that the incoming syntax returns a valid JSONObject or array.

Implementation principle

Before implementation, a basic grammar needs to be defined, which mainly supports the following four usages:

• Single key query: Get(json,"name")
• Nested query: Get(json,"obj1.obj2.obj3.name")
• Array query: Get(json,"obj.array[0]")
• Array nested query: Get(json,"obj.array[0].obj2.obj3[1].name")

The grammar is very simple and conforms to the grammar rules we encounter every day, so that we can access any value in the JSON data.

In fact, the implementation process is not complicated. We have already converted the JSON string into a JSONObject in the previous article.

This time, it is just to parse the grammar just defined as token , and then parse the token JSONObject .

Finally, after parsing token , the data obtained JSONObject can be returned.

---

Let's take this query code as an example:

The first step is to perform lexical analysis on the query syntax, and finally get the following figure token .

Simple syntax checking can also be done during lexical analysis; for example, if an array query is included, a syntax error will be thrown when it does not end with the ] symbol.

Then we iterate over the tokens of the grammar. As shown below:

Whenever it traverses to token type is Key , it will get data from the current JSONObject, and overwrite the current JSONObject with the obtained value.

其中每当遇到---cbe6e18e297cd676392f7b331c850bb4 . [ ]的token 时便消耗掉，直到我们将token 遍历完毕，这时将当前JSONObject just go back.

During the traversal process, when an illegal format is encountered, such as ---5826235bb4ba5339691b92b3514341c0---, an empty JSONObject obj_list[1.] will be returned.

Grammar verification is actually very easy to do, because according to our grammar rules, Array in index must be followed by a EndArray , As long as it's not one EndArray the syntax is illegal.

If you are interested, you can look at the source code of the parsing process:

https://github.com/crossoverJie/gjson/blob/cfbca51cc9bc0c77e6cb9c9ad3f964b2054b3826/json.go#L46

Do four operations on JSON

 str := `{"name":"bob", "age":10,"magic":10.1, "score":{"math":[1,2]}}`
    result := GetWithArithmetic(str, "(age+age)*age+magic")
    assert.Equal(t, result.Float(), 210.1)
    result = GetWithArithmetic(str, "(age+age)*age")
    assert.Equal(t, result.Int(), 200)

    result = GetWithArithmetic(str, "(age+age) * age + score.math[0]")
    assert.Equal(t, result.Int(), 201)

    result = GetWithArithmetic(str, "(age+age) * age - score.math[0]")
    assert.Equal(t, result.Int(), 199)

    result = GetWithArithmetic(str, "score.math[1] / score.math[0]")
    assert.Equal(t, result.Int(), 2)

Finally, I also extended the syntax to support JSON the shaping in the data （int、float） to do four arithmetic operations, although this is a niche demand, but I think it is quite interesting to finish it. , I haven't found a library with similar functions on the market at present, which may be related to niche needs 🤣.

The core four arithmetic logics are provided by the script interpreter written before:

https://github.com/crossoverJie/gscript

A function is provided separately, and a four arithmetic expression is passed in to return the calculation result.

Since the previous version did not support float, it was specially adapted this time.

Due to space limitations, more implementation logic of these four operations will continue to be shared later.

Summarize

So far, this is the first time that I have used the knowledge of compilation principles to solve a problem in a specific field. I have always felt that the compilation principles are relatively advanced in college and work, so I have always resisted, but after this period of study and practice, I gradually Also got a little bit of a doorway.

However, it is only the tip of the iceberg at present. The back end of the compilation principle involves the underlying knowledge of the computer, so there is still a long way to go.

The above are all off-topic, and I will maintain this library for a long time; in order to meet the production requirements, I try to improve the single test coverage rate, which is currently 98%.

You are also welcome to use and file bugs🐛.

We will continue to optimize later, such as supporting escape characters, improving performance, etc.

Interested friends, please continue to pay attention to:
https://github.com/crossoverJie/gjson