CODEWITHSUNDEEP
c#.netencodingf#
App2015
App2015

Reputation: 993

Http request ending as bad request due to Encoding and Escape character issues

Note: Code is in F# but tagging C# because it's a general encoding and escape character issue across .net while converting from object to json, please look at the output at the bottom of the page.

Following request is ending up a bad request, please have a look at any discrepancies especially in the Encoding.Default.GetString and then encoding back into System.Text.Encoding.ASCII.GetBytes especially.

Context: An object is serialized and written to the body of the PUT request and is ended up as bad request 400. JSON is serialized correctly and I can see the string with Console.WriteLine

I took the JSON encoding example from F# snippets

let update (schema: Article) : Article = 

    let url = "http://example.com"

    use memoryStream = new MemoryStream() 
    (new DataContractJsonSerializer(typeof<Article>)).WriteObject(memoryStream, schema) 
    let d = Encoding.Default.GetString(memoryStream.ToArray()) // I suspect problem may be here
    let data : byte[] = System.Text.Encoding.ASCII.GetBytes(d); // or here

    let request = WebRequest.Create(url) :?> HttpWebRequest
    request.Method <- "PUT"
    request.ContentType <- "application/json;charset=UTF-8"
    request.Accept <- "application/json;charset=UTF-8"
    request.ContentLength <- (int64)data.Length

    use requestStream = request.GetRequestStream() 
    requestStream.Write(data, 0, data.Length)
    requestStream.Flush()
    requestStream.Close()

    let response = request.GetResponse() :?> HttpWebResponse
    use reader = new StreamReader(response.GetResponseStream())
    use memoryStream = new MemoryStream(ASCIIEncoding.Default.GetBytes(reader.ReadToEnd())) 
    let result = (new DataContractJsonSerializer(typeof<Article>)).ReadObject(memoryStream) :?> Article

    reader.Close()
    response.Close()
    request.Abort()

    result

Further Discoveries

I was able to get Charles Proxy up and I saw that forward slashes have been escaped in the output serialized json to the server.

Actual Output: with escaped slashes \ /

\/publication\/d40a4e4c-d6a3-45ae-98b3-924b31d8712a\/altasset\/c42d0df7-a563-4bb5-b7fa-313e6a98032f\/

Expected output: They are supposed to have forward slashes only, no escaping of the characters.

/publication/d40a4e4c-d6a3-45ae-98b3-924b31d8712a....

I guess something causing the character to be escaped in Encoding process, is there a way to fix it?

Edit

I also noticed that memoryStream.ToArray() returns a byte[] so instead of going through encodings I also tried the following but it didn't make any difference.

let data : byte[] = memoryStream.ToArray()

requestStream.Write(data, 0, data.Length)

in Charles proxy, I'm seeing Broken pipe message.

enter image description here

Upvotes: 2

Views: 885

Answers (1)

Jono
Jono

Reputation: 2054

Add a reference to the Newtonsoft.Json NuGet package and add this to the top of your F# file:

open Newtonsoft.Json

Then try this as the body of your update method (it uses UTF-8 encoding explicitly rather than ASCII or the platform default):

let url = "http://example.com"

let serialized = JsonConvert.SerializeObject(schema)
let data : byte[] = Encoding.UTF8.GetBytes(serialized)

let request = WebRequest.Create(url) :?> HttpWebRequest
request.Method <- "PUT"
request.ContentType <- "application/json;charset=UTF-8"
request.Accept <- "application/json;charset=UTF-8"
request.ContentLength <- (int64)data.Length

use requestStream = request.GetRequestStream() 
requestStream.Write(data, 0, data.Length)
requestStream.Flush()
requestStream.Close()

let response = request.GetResponse() :?> HttpWebResponse
use reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8)
let result = JsonConvert.DeserializeObject<Article>(reader.ReadToEnd())

reader.Close()
response.Close()
request.Abort()

result

Unless the server you're using was also using DataContractJsonSerializer (which escapes "/" as "\/") then you're probably better off with Newtonsoft. And it doesn't hurt to fix the UTF-8 encoding issues before they arise (it didn't look like you were sending/receiving any non-ASCII text but that would inevitably happen in the real world).

Upvotes: 1

Related Questions