mjhd
mjhd

Reputation: 65

Golang slow when looping over Collection

Im feeding my app a json from Redis which I then unmarshal and loop through.

Here is what the json Im feeding from Redis looks like:

[
    {
        "titel": "test 1",
        "event": "some value",
        "pair": "some value",
        "condition": [
            "or",
            [
                "contains",
                "url",
                "/"
            ],[
                "notcontains",
                "url",
                "hello"
            ]
        ],
        "actions": [
            [
                "option1",
                "12",
                "1"
            ],
            [
                "option2",
                "3",
                "1"
            ]
        ]
    }, {
        "titel": "test 2",
        "event": "some value",
        "pair": "some value",
        "condition": [
            "or",
            [
                "contains",
                "url",
                "/"
            ]
        ],
        "actions": [
            [
                "option1",
                "12",
                "1"
            ],
            [
                "option2",
                "3",
                "1"
            ]
        ]
    }
]

My struct to store the json looks like this:

type Trigger struct {
    Event     string        `json:"event"`  
    Pair      string        `json:"pair"`   
    Actions   [][]string    `json:"actions"`
    Condition Condition     `json:"condition"`
}

type Condition []interface{}

func (c *Condition) Typ() string {
    return (*c)[0].(string)
}

func (c *Condition) Val() []string {
    xs := (*c)[1].([]interface{})
    ys := make([]string, len(xs))
    for i, x := range xs {
        ys[i] = x.(string)
    }
    return ys
}

func (c *Condition) String() (string, string, string) {
    return c.Val()[0], c.Val()[1], c.Val()[2]
}

type Triggers struct {
    Collection []Trigger
}

And Im unmarshaling it into the struct like so:

var triggers Triggers
err := json.Unmarshal([]byte(triggersJson), &triggers.Collection)
if err != nil {
    fmt.Println("Error while unmarshaling triggers json: ", err)
}

This works and performs perfectly fine while testing it with siege. However, as soon as I want to start loop over the struct Im starting to see long response times and timeouts.

This is how I loop over it:

for _,element := range triggers.Collection {
    if element.Event == "some value" {
        fmt.Println("We got: some value")
    }
 }

Performance without the loop is 2ms on 500 concurrent connections. With the loop its 600ihs ms on 500 concurrent with a bunch of timeouts

Ideally Id like to change structure of the json to not include:

"or",

But Im not in control over the json so this is unfortunately impossible.

Any ideas whats causing this and how it could be resolved?

Edit

I found whats slowing the whole thing down.

Editing my struct like:

type Trigger struct {
    Event     string        `json:"event"`  
    Pair      string        `json:"pair"`   
    Actions   [][]string    `json:"actions"`
    //Condition Condition     `json:"condition"`
}

Takes it from 600ms to 100ms. However now Im not able to parse the Condition.

Now sure how to parse the Condition in another way than Im currently doing due it having two different formats

Upvotes: 1

Views: 456

Answers (1)

elgris
elgris

Reputation: 526

Sounds weird, but try to avoid copying Collection elements. Something like this:

for i := range triggers.Collection {
    if triggers.Collection[i].Event == "some value" {
        fmt.Println("We got: some value")
    }

Upvotes: 4

Related Questions