How do I send multipart form data in an HTTP request (for Watson NLC training)?

Question

I'm writing application that'll use Watson Natural Language Classifier ("NLC"). When I send an HTTP POST request against the v1/classifiers URI path with the following request message body, the server responds with status code 415 (Unsupported Media Type):

--04fef47728eb08148fe9c7b18dd42b75abd75ebf752fd3412a85aa3af075
Content-Disposition: form-data; name="training_data"; filename="data.csv"
Content-Type: text/csv

How hot is it today?;temperature
Is it hot outside?;temperature
Will it be uncomfortably hot?;temperature
Will it be sweltering?;temperature
How cold is it today?;temperature
Is it cold outside?;temperature
Will it be uncomfortably cold?;temperature
Will it be frigid?;temperature
What is the expected high for today?;temperature
What is the expected temperature?;temperature
Will high temperatures be dangerous?;temperature
Is it dangerously cold?;temperature
When will the heat subside?;temperature
Is it hot?;temperature
Is it cold?;temperature
How cold is it now?;temperature
Will we have a cold day today?;temperature
When will the cold subside?;temperature
What highs are we expecting?;temperature
What lows are we expecting?;temperature
Is it warm?;temperature
Is it chilly?;temperature
What's the current temp in Celsius?;temperature
What is the temperature in Fahrenheit?;temperature
--04fef47728eb08148fe9c7b18dd42b75abd75ebf752fd3412a85aa3af075
Content-Disposition: form-data; name="training_metadata"; filename="metadata.json"
Content-Type: application/json

{"language": "en"}

The 415 status code suggests a problem with the content type, but it seems like all the MIME types are correct.

My code (written in Go):

func (w WatsonClassifier) createFormFile(writer *multipart.Writer, fieldname string, filename string, contentType string) (io.Writer, error)     {
    h := make(textproto.MIMEHeader)
    h.Set("Content-Disposition",
        fmt.Sprintf(`form-data; name="%s"; filename="%s"`,
            fieldname, filename))
    h.Set("Content-Type", contentType)
    return writer.CreatePart(h)
}

func (w WatsonClassifier) request(method string, apiUrl string, body io.Reader) (string, error) {
    url := w.url + "/" + apiUrl
    req, err := http.NewRequest(method, url, body)
    if err != nil {
        return "", err
    }
    req.SetBasicAuth(w.username, w.password)
    client := http.Client{}
    resp, err := client.Do(req)
    if resp.StatusCode != 200 {
        answer, _ := ioutil.ReadAll(resp.Body)
        fmt.Println(string(answer))
        return "", errors.New("Watson returned wrong status code : " + resp.Status)
    }
    answer, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }
    return string(answer), nil
}

func (w WatsonClassifier) Train(data []ClassifierCategory) (Classifier, error) {
    table := w.buildTable(data)
    str := w.buildCsv(data)
    buf := new(bytes.Buffer)
    writer := multipart.NewWriter(buf)
    data_part, err := w.createFormFile(writer, "training_data", "data.csv", "text/csv")
    if err != nil {
        return WatsonClassifier{}, err
    }
    data_part.Write([]byte(str))
    metadata_part, err := w.createFormFile(writer, "training_metadata", "metadata.json", "application/json")
    if err != nil {
        return WatsonClassifier{}, err
    }
    metadata_json := "{\"language\": \"" + w.Language + "\"}"
    metadata_part.Write([]byte(metadata_json))
    fmt.Println(buf.String())
    answer, err := w.request("POST", "v1/classifiers", buf)
    if err != nil {
        return WatsonClassifier{}, err
    }
    fmt.Println(answer)
    return WatsonClassifier{}, nil
}

Answer 1

Note that curl is sending a "Content-Type" header of multipart/form-data. Your Go program is sending a "Content-Disposition" header with just form-data (note the difference, it lacking the leading multipart composite top-level media type), but it doesn't take care of sending the correct "Content-Type" header for the containing HTTP request.

Go's multipart.Writer type's CreateFormFile method does the same, but again, that's only part of the job:

h.Set("Content-Disposition",
        fmt.Sprintf(`form-data; name="%s"; filename="%s"`,
                escapeQuotes(fieldname), escapeQuotes(filename)))

To get the proper "Content-Type" header value, you need to use multipart.Writer.FormDataContentType. To put that value to use, you'll need to get your multipart.Writer into your WatsonClassifier.request method, so that you can set the content type on your http.Request instance:

req.Header.Set("Content-Type", writer.FormDataContentType())

Alternately, add another parameter to WatsonClassifier.request for the content type, and pass the result of FormDataContentType as the argument from the call site in WatsonClassifier.Train.

Let us know if that does the trick.