CODEWITHSUNDEEP
httputf-8character-encodingnodemcu
S. Blanc
S. Blanc

Reputation: 107

NodeMCU http.get() and UTF-8 charset

I am using the http module of the NodeMCU dev branch to make a GET request to the Google Calendar API. However, when I retrieve the events and parse answer, I get strange chars due to a non-proper encoding.

I tried to add Accept-Charset: utf-8 in the header of the request, but the request fails (code=-1).

Is there a way to set the charset, or convert it afterwards in lua? 

function bdayshttps(curr_date)
    if (string.len(curr_date) == 10) then
        http.get("https://www.googleapis.com/calendar/v3/calendars/"..
                    "<CalendarID>/events"..
                    "?timeMax="..curr_date.."T23%3A59%3A59-00%3A00"..
                    "&timeMin="..curr_date.."T00%3A00%3A00-00%3A00&fields=items%2Fsummary"..
                    "&key=<Google Calendar API key>", "Accept-Charset: utf-8", function(code, data)
            if (code < 0) then
              print("msg:birthdays error")
            else
              if (code == 200) then
                output = ""
                for line in data:gmatch"\"summary\": \"[^\n]*" do
                    output = output..line:sub(13, line:len()-1)..";"
                end
                print("bday:"..output)
              end
            end
        end)
    end
end

For obvious reasons, I erased the calendarID and API key.

EDIT:

The result of this code returns msg:birthday error, meaning the GET request returns a code=-1.

When replacing the "Accept-Charset: utf-8" by nil in the header, I get:

Loïc Simonetti instead of Loïc Simonetti.

Upvotes: 1

Views: 1516

Answers (2)

Adam B
Adam B

Reputation: 3833

For the second part of your question (now that the response is working), it appears you are receiving back valid UTF8 Unicode.

Loïc corresponds to the following UTF8 code units: 4C 6F C3 AF 63

Loïc has the following byte values in a common encoding (CodePage 1252): 4C 6F C3 AF 63.

So my guess is the terminal or other device you're using to view the bytes is mangling things as opposed to your request/response being incorrect. Per @Marcel's link above too, Lua does not handle Unicode at all so this is about the best you can do (safely receive, store, and then send it back later)

Upvotes: 1

Marcel St&#246;r
Marcel St&#246;r

Reputation: 23535

The API docs say that you need to append \r\n to every header you set. There's an example in the docs for http.post().

Hence, instead of "Accept-Charset: utf-8" you should set "Accept-Charset: utf-8\r\n".

Upvotes: 3

Related Questions