How to retrieve form-data as map (like PHP and Ruby) in Go (Golang)

Question

I'm a PHP Dev. But currently moving to Golang... I'm trying to retrieve data from a Form (Post method):

<!-- A really SIMPLE form -->
<form class="" action="/Contact" method="post">
  <input type="text" name="Contact[Name]" value="Something">   
  <input type="text" name="Contact[Email]" value="Else">
  <textarea name="Contact[Message]">For this message</textarea>
  <button type="submit">Submit</button>
</form>

In PHP I would simple use this to get the data:

<?php 
   print_r($_POST["Contact"])
?>
// Output would be something like this:
Array
(
    [Name] => Something
    [Email] => Else
    [Message] => For this message
)

BUT in go... either I get one by one or the whole thing but not the Contact[] Array only such as PHP

I thought about 2 solutions:

1) Get one by one:

// r := *http.Request
err := r.ParseForm()

if err != nil {
    w.Write([]byte(err.Error()))
    return
}

contact := make(map[string]string)

contact["Name"] = r.PostFormValue("Contact[Name]")
contact["Email"] = r.PostFormValue("Contact[Email]")
contact["Message"] = r.PostFormValue("Contact[Message]")

fmt.Println(contact)

// Output
map[Name:Something Email:Else Message:For this Message]

Note that the map keys are the whole: "Contact[Name]"...

2) Range whole map r.Form and "parse|obtain" those values with Prefix "Contact[" and then replacing "Contact[" and "]" with empty string so I can get the Form array Key only such the PHP Example

I went for this work around by my own but... ranging over the whole form may not be a good idea (?)

// ContactPost process the form sent by the user
func ContactPost(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    err := r.ParseForm()

    if err != nil {
        w.Write([]byte(err.Error()))
        return
    }

    contact := make(map[string]string)

   for i := range r.Form {
       if strings.HasPrefix(i, "Contact[") {
           rp := strings.NewReplacer("Contact[", "", "]", "")
           contact[rp.Replace(i)] = r.Form.Get(i)
       }
   }

    w.Write([]byte(fmt.Sprint(contact)))
}
//Output
map[Name:Something Email:Else Message:For this Message]

Both solutions give me the same output... But in the 2nd example I don't necessarily need to know the keys of "Contact[]"

I know... I may just forget about that "Form Array" and use name="Email" on my inputs and retrieve one by one but... I've passing through some scenarios where I use ONE form that contain more than 2 arrays of data and do different things with each one, like ORMs

Question 1: Is there a easier way to get my Form Array as an actual map in Golang like PHP does?

Question 2: Should I retrieve the data one by one (Tedious as much and I may change the Form data at some point and recompile...) or iterate the whole thing as I've done in the 2nd example.

Sorry for my bad English... Thanks in advance!

Answer 1

I wrote some code, that transforms FormData array to json string.

package phprubyformdatatojson

import (
    "bytes"
    "io"
    "net/url"
    "strconv"
    "strings"
)

type Node struct {
    Name       string
    Value      string
    Subnodes   []*Node
    ArrayValue []*Node
}

func getJsonFromNode(rootNode *Node) string {
    return "{" + nodeToJson(rootNode) + "}"
}

func nodeToJson(n *Node) string {
    if len(n.Subnodes) == 0 && len(n.ArrayValue) == 0 {
        return "\"" + n.Name + "\"" + ": " + "\"" + n.Value + "\""
    }

    if len(n.Subnodes) > 0 {
        var parts []string

        for _, subnode := range n.Subnodes {
            parts = append(parts, nodeToJson(subnode))
        }

        if len(n.Name) > 0 {
            return "\"" + n.Name + "\"" + ": {" + strings.Join(parts, ", ") + "}"

        } else {
            return strings.Join(parts, ", ")
        }
    }

    if len(n.ArrayValue) > 0 {
        var parts []string
        for _, arrayPart := range n.ArrayValue {
            parts = append(parts, "{"+nodeToJson(arrayPart)+"}")
        }
        return "\"" + n.Name + "\"" + ": [" + strings.Join(parts, ", ") + "]"
    }

    return "{}"
}

func addNode(nodeMap map[string]*Node, key string, value string) map[string]*Node {
    keys := splitKeyToParts(key)
    var lastNode *Node
    previosKey := "rootNode"
    totalKey := ""
    for index, keyPart := range keys {
        if totalKey == "" {
            totalKey += keyPart
        } else {
            totalKey += "|||" + keyPart
        }
        isNumber := false
        if _, err := strconv.Atoi(keyPart); err == nil {
            isNumber = true
        }
        if index < len(keys)-1 {
            if z, ok := nodeMap[totalKey]; !ok {
                lastNode = z
                node := &Node{}
                nodeMap[totalKey] = node
                lastNode = node
                prevNode, oook := nodeMap[previosKey]
                if oook {
                    if isNumber {
                        prevNode.ArrayValue = append(prevNode.ArrayValue, node)
                    } else {
                        node.Name = keyPart
                        prevNode.Subnodes = append(prevNode.Subnodes, node)
                    }
                }
            }
        } else {
            lastNode = nodeMap[previosKey]
            newNode := &Node{Name: keyPart, Value: value}
            if isNumber {
                lastNode.ArrayValue = append(lastNode.ArrayValue, newNode)
            } else {
                lastNode.Subnodes = append(lastNode.Subnodes, newNode)
            }
        }
        previosKey = totalKey

    }
    return nodeMap
}

func splitKeyToParts(key string) []string {
    const DELIMITER = "|||||"
    key = strings.Replace(key, "][", DELIMITER, -1)
    key = strings.Replace(key, "[", DELIMITER, -1)
    key = strings.Replace(key, "]", DELIMITER, -1)
    key = strings.Trim(key, DELIMITER)

    return strings.Split(key, DELIMITER)
}

func TransformMapToJsonString(source map[string][]string) string {
    nodesMap := map[string]*Node{}
    nodesMap["rootNode"] = &Node{}

    for key, value := range source {
        nodesMap = addNode(nodesMap, key, strings.Join(value, ""))

    }
    return getJsonFromNode(nodesMap["rootNode"])
}

When you can manualy transform you request and json.Unmarshal it, or write gin.Middleware

func PhpRubyArraysToJsonMiddleware(c *gin.Context) {
    body, _ := c.GetRawData()
    m, _ := url.ParseQuery(string(body))
    parsedJson := TransformMapToJsonString(m)
    newBody := []byte(parsedJson)
    c.Request.Body = io.NopCloser(bytes.NewBuffer(newBody))

    c.Next()
}

and use it like this

func handelUpdate(c *gin.Context) {

    req := &YourJsonStruct{}
    if err := c.BindJSON(req); err != nil {
        c.Status(http.StatusBadRequest)
        return
    }

    // your code
}
func main() {
  router := gin.Default()
  router.Use(PhpRubyArraysToJsonMiddleware)
  router.POST("/update", handelUpdate)
}

Answer 2

I've been using the dot prefix convention: contact.name, contact.email

I decided to leave a script here so people don't have to spend so much time writing their own custom parser.

Here is a simple script that traverses the form data and puts the values in a struct that follows a format close to PHP and Ruby's.

package formparser

import (
    "strings"
    "mime/multipart"
)

type NestedFormData struct {
    Value *ValueNode
    File *FileNode
}

type ValueNode struct {
    Value []string
    Children map[string]*ValueNode
}

type FileNode struct {
    Value []*multipart.FileHeader
    Children map[string]*FileNode
}

func (fd *NestedFormData) ParseValues(m map[string][]string){
    n := &ValueNode{
        Children: make(map[string]*ValueNode),
    }
    for key, val := range m {
        keys := strings.Split(key,".")
        fd.nestValues(n, &keys, val)
    }
    fd.Value = n
}

func (fd *NestedFormData) ParseFiles(m map[string][]*multipart.FileHeader){
    n := &FileNode{
        Children: make(map[string]*FileNode),
    }
    for key, val := range m {
        keys := strings.Split(key,".")
        fd.nestFiles(n, &keys, val)
    }
    fd.File = n
}

func (fd *NestedFormData) nestValues(n *ValueNode, k *[]string, v []string) {
    var key string
    key, *k = (*k)[0], (*k)[1:]
    if len(*k) == 0 {
            if _, ok := n.Children[key]; ok {
                    n.Children[key].Value = append(n.Children[key].Value, v...)
            } else {
                    cn := &ValueNode{
                            Value: v,
                            Children: make(map[string]*ValueNode),
                    }
                    n.Children[key] = cn
            }
    } else {
        if _, ok := n.Children[key]; ok {
            fd.nestValues(n.Children[key], k,v)
        } else {
            cn := &ValueNode{
                Children: make(map[string]*ValueNode),
            }
            n.Children[key] = cn
            fd.nestValues(cn, k,v)
        }
    }
}

func (fd *NestedFormData) nestFiles(n *FileNode, k *[]string, v []*multipart.FileHeader){
    var key string
    key, *k = (*k)[0], (*k)[1:]
    if len(*k) == 0 {
        if _, ok := n.Children[key]; ok {
            n.Children[key].Value = append(n.Children[key].Value, v...)
        } else {
            cn := &FileNode{
                Value: v,
                Children: make(map[string]*FileNode),
            }
            n.Children[key] = cn
        }
    } else {
        if _, ok := n.Children[key]; ok {
            fd.nestFiles(n.Children[key], k,v)
        } else {
            cn := &FileNode{
                Children: make(map[string]*FileNode),
            }
            n.Children[key] = cn
            fd.nestFiles(cn, k,v)
        }
    }
}

Then you can use the package like so:

package main

import (
 "MODULE_PATH/formparser"
 "strconv"
 "fmt"
)

func main(){
    formdata := map[string][]string{
        "contact.name": []string{"John Doe"},
        "avatars.0.type": []string{"water"},
        "avatars.0.name": []string{"Korra"},
        "avatars.1.type": []string{"air"},
        "avatars.1.name": []string{"Aang"},
    }
    f := &formparser.NestedFormData{}
    f.ParseValues(formdata)
    //then access form values like so
    fmt.Println(f.Value.Children["contact"].Children["name"].Value)
    fmt.Println(f.Value.Children["avatars"].Children[strconv.Itoa(0)].Children["name"].Value)
    fmt.Println(f.Value.Children["avatars"].Children[strconv.Itoa(0)].Children["type"].Value)
    fmt.Println(f.Value.Children["avatars"].Children[strconv.Itoa(1)].Children["name"].Value)
    fmt.Println(f.Value.Children["avatars"].Children[strconv.Itoa(1)].Children["type"].Value)
    //or traverse  the Children in a loop
    for key, child := range f.Value.Children {
        fmt.Println("Key:", key, "Value:", child.Value)
        if child.Children != nil {
            for k, c := range child.Children {
                fmt.Println(key + "'s child key:", k, "Value:", c.Value)
            }
        }
    }
    //if you want to access files do not forget to call f.ParseFiles()
}

Answer 3

I had a similar problem, so I wrote this function

func ParseFormCollection(r *http.Request, typeName string) []map[string]string {
    var result []map[string]string
    r.ParseForm()
    for key, values := range r.Form {
        re := regexp.MustCompile(typeName + "\\[([0-9]+)\\]\\[([a-zA-Z]+)\\]")
        matches := re.FindStringSubmatch(key)

        if len(matches) >= 3 {

            index, _ := strconv.Atoi(matches[1])

            for ; index >= len(result); {
                result = append(result, map[string]string{})
            }

            result[index][matches[2]] = values[0]
        }
    }
    return result
}

It turns a collection of form key value pairs into a list of string maps. For example, if I have form data like this:

Contacts[0][Name] = Alice
Contacts[0][City] = Seattle
Contacts[1][Name] = Bob
Contacts[1][City] = Boston

I can call my function passing the typeName of "Contacts":

for _, contact := range ParseFormCollection(r, "Contacts") {
    // ...
}

And it will return a list of two map objects, each map containing keys for "Name" and "City". In JSON notation, it would look like this:

[
  {
    "Name": "Alice",
    "City": "Seattle"
  },
  {
    "Name": "Bob",
    "City": "Boston"
  }
]

Which incidentally, is exactly how I'm posting the data up to the server in an ajax request:

$.ajax({
  method: "PUT",
  url: "/api/example/",
  dataType: "json",
  data: {
    Contacts: [
      {
        "Name": "Alice",
        "City": "Seattle"
      },
      {
        "Name": "Bob",
        "City": "Boston"
      }
    ]
  }
})

If your form data key structure doesn't quite match mine, then I you could probably adapt the Regex that I'm using to suit your needs.

Answer 4

I had the same question. The submission of array form params is also idiomatic in the Ruby/Rails world where I'm coming from. But, after some research, it looks like this is not really the "Go-way".

I've been using the dot prefix convention: contact.name, contact.email, etc.

func parseFormHandler(writer http.ResponseWriter, request *http.Request) {
    request.ParseForm()

    userParams := make(map[string]string)

    for key, _ := range request.Form {
        if strings.HasPrefix(key, "contact.") {
            userParams[string(key[8:])] = request.Form.Get(key)
        }
    }

    fmt.Fprintf(writer, "%#v\n", userParams)
}

func main() {
    server := http.Server{Addr: ":8088"}
    http.HandleFunc("/", parseFormHandler)
    server.ListenAndServe()
}

Running this server and then curling it:

$ curl -id "contact.name=Jeffrey%20Lebowski&contact.email=thedude@example.com&contact.message=I%20hate%20the%20Eagles,%20man." http://localhost:8088

Results in:

HTTP/1.1 200 OK
Date: Thu, 12 May 2016 16:41:44 GMT
Content-Length: 113
Content-Type: text/plain; charset=utf-8

map[string]string{"name":"Jeffrey Lebowski", "email":"thedude@example.com", "message":"I hate the Eagles, man."}

Using the Gorilla Toolkit

You can also use the Gorilla Toolkit's Schema Package to parse the form params into a struct, like so:

type Submission struct {
    Contact Contact
}

type Contact struct {
    Name    string
    Email   string
    Message string
}

func parseFormHandler(writer http.ResponseWriter, request *http.Request) {
    request.ParseForm()

    decoder := schema.NewDecoder()
    submission := new(Submission)
    err := decoder.Decode(submission, request.Form)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Fprintf(writer, "%#v\n", submission)
}

Running this server and then curling it:

$ curl -id "Contact.Name=Jeffrey%20Lebowski&Contact.Email=thedude@example.com&Contact.Message=I%20hate%20the%20Eagles,%20man." http://localhost:8088

Results in:

HTTP/1.1 200 OK
Date: Thu, 12 May 2016 17:03:38 GMT
Content-Length: 128
Content-Type: text/plain; charset=utf-8

&main.Submission{Contact:main.Contact{Name:"Jeffrey Lebowski", Email:"thedude@example.com", Message:"I hate the Eagles, man."}}

Answer 5

Is there a easier way to get my Form Array as an actual map in Golang like PHP does?

You can use the PostForm member of the http.Request type. It is of type url.Values -- which is actually (ta-da) a map[string][]string, and you can treat is as such. You'll still need to call req.ParseForm() first, though.

if err := req.ParseForm(); err != nil {
    // handle error
}

for key, values := range req.PostForm {
    // [...]
}

Note that PostForm is a map of lists of strings. That's because in theory, each field could be present multiple times in the POST body. The PostFormValue() method handles this by implicitly returning the first of multiple values (meaning, when your POST body is &foo=bar&foo=baz, then req.PostFormValue("foo") will always return "bar").

Also note that PostForm will never contain nested structures like you are used from PHP. As Go is statically typed, a POST form value will always be a mapping of string (name) to []string (value/s).

Personally, I wouldn't use the bracket syntax (contact[email]) for POST field names in Go applications; that's a PHP specific construct, anyway and as you've already noticed, Go does not support it very well.

Should I retrieve the data one by one (Tedious as much and I may change the Form data at some point and recompile...) or iterate the whole thing as I've done in the 2nd example.

There's probably no correct answer for that. If you are mapping your POST fields to a struct with static fields, you'll have to explicitly map them at some point (or use reflect to implement some magical auto-mapping).