golang: How can I populate a multi-struct map in a loop?

Question

I have log files of customer interactions with an API. I want to parse those logs and feed the results into a map of structs so that I can organize the data into helpful information. For example, I would like to respond to the following query: "show me the total number of requests per user per day".

I have created what seems like an adequate structure to hold the data. However, when I try to run the program I get the error: invalid operation: dates[fields[1]] (type *Dates does not support indexing) [process exited with non-zero status].

http://play.golang.org/p/8u3jX26ktt

package main

import (
    "fmt"
    "strings"
)

type Stats struct {
    totalNumberOfRequests int
}
type Customer struct {
    listOfCustomers map[string]Stats // map[customerid]Stats
}
type Dates struct {
    listOfDates map[string]Customer // map[date]Customer
}

var requestLog = []string{
    "2011-10-05, 1234, apiquery",
    "2011-10-06, 1234, apiquery",
    "2011-10-06, 5678, apiquery",
    "2011-10-09, 1234, apiquery",
    "2011-10-12, 1234, apiquery",
    "2011-10-13, 1234, apiquery",
}

func main() {
    dates := new(Dates)
    for _, entry := range requestLog {
        fmt.Println("entry:", entry)
        fields := strings.Split(entry, "'")
        dates.listOfDates[fields[0]].listOfCustomers[fields[1]].totalNumberOfRequests++
    }
}

Is there a better structure to use? Or is there a way to make this structure work for this particular purpose?

Answer 1

If I understood your expectations about the output here's a solution. However I don't like that "Customer is a map with id and Stat.. I think it should be a simpler struct with two Fields (cid string and stat Stats). Also the Dates structure does not allow for multiple customers on the same date, so I've changed to map single date to list of users.

I've also added more "tests scenarios" to cover the cases for a customer accessing the resource multiple times on the same date.

You don't seem to use the "apiquery" your example, so the code below does not match against it.

Regarding the change to pointers in the structs - see this issue (as noted in the comments to your question)

package main

import (
    "fmt"
    "strings"
)

type Stats struct {
    totalNumberOfRequests int
}
type Customer struct {
    customerWithStat map[string]*Stats // a customer with it's corresponding stats
}

type Dates struct {
    listOfDates map[string][]*Customer // map[date]list of customers (for each date)
}

var requestLog = []string{
    "2011-10-05, 1234, apiquery",
    "2011-10-06, 5678, apiquery",
    "2011-10-06, 1234, apiquery",
    "2011-10-06, 1234, apiquery",
    "2011-10-06, 5678, apiquery",
    "2011-10-06, 1234, apiquery",
    "2011-10-09, 1234, apiquery",
    "2011-10-12, 1234, apiquery",
    "2011-10-13, 1234, apiquery",
    "2011-10-13, 1234, apiquery",
    "2011-10-06, 1234, apiquery",
}

func main() {
    listOfDates := make(map[string][]*Customer)
    dates := Dates{listOfDates}
    for _, entry := range requestLog {
        fields := strings.Split(entry, ",")
        curDateStr := strings.TrimSpace(fields[0])
        curCustIdStr := strings.TrimSpace(fields[1])

        if customersAtDate, dateExists := dates.listOfDates[curDateStr]; dateExists {
            // Date already exist
            for _, curCustomer := range customersAtDate {
                if curStat, customerExists := curCustomer.customerWithStat[curCustIdStr]; customerExists {
                    // The user has already accessed this resource - just increment
                    curStat.totalNumberOfRequests++
                } else {
                    // New user - set access to 1
                    curCustomer.customerWithStat[curCustIdStr] = &Stats{1}
                }
            }
        } else {
            // New Date

            // Init the Statistic for the new customer
            newCustomerData := make(map[string]*Stats)
            newCustomerData[curCustIdStr] = &Stats{1}

            // Create the customer itself
            newCustomer := &Customer{newCustomerData}

            // add to the current day list
            dates.listOfDates[curDateStr] = append(dates.listOfDates[curDateStr], newCustomer)

        }
    }

    // Print result
    for date, customers := range dates.listOfDates {
        fmt.Println("Date: ", date)
        for _, customer := range customers {
            for cid, stat := range customer.customerWithStat {
                fmt.Println("  Customer: ", cid)
                fmt.Println("  # Requests: ", *stat)
            }
        }
    }
}

This will output:

Date:  2011-10-05
  Customer:  1234
  # Requests:  {1}
Date:  2011-10-06
  Customer:  5678
  # Requests:  {2}
  Customer:  1234
  # Requests:  {4}
Date:  2011-10-09
  Customer:  1234
  # Requests:  {1}
Date:  2011-10-12
  Customer:  1234
  # Requests:  {1}
Date:  2011-10-13
  Customer:  1234
  # Requests:  {2}