CODEWITHSUNDEEP
node.jscharacter-encodingrequestsails.js
Valentin Chtoutz
Valentin Chtoutz

Reputation: 1

Submit POST form with latin1 charset in Node.js

Short version of the problem : in my Node.js app using Sails.js framework, I'd like to send data (simple UTF-8 text charset) to a distant server POST form, which has ISO-8859-15 charset.

Details : using the request package, the form is submitted this way :

var postFields = {
 'message'  : message,  // <== this var contains chars such as é,à,ç, etc.
 'submit'           : 'Send'
};
var opts = {
 uri: url,
 headers: {
  'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:33.0) Gecko/20100101 Firefox/33.0',
  'Cookie'    : cookie,
  'Content-type': "application/x-www-form-urlencoded; charset=ISO-8859-15"
 },
 form: postFields
};
request.post(opts, function(error, response, body) {
 if (!error && response.statusCode == 200) {
   return callback(null, true);
 } else {
   return callback(null, false);
 }
});

This works fine, my form is sent successfully, except the "é" chars turned into ugly "é" to the distant server. In the 'Content-type' property of the opts object, I tried to set charset to ISO-8859-15 / UTF-8 / none, text/html or application/x-www-form-urlencoded, still doesn't work...

Don't know if this helps, but here is what I have to do :

1) get data from the distant server (latin1 encoded), then convert it to UTF-8 text :

var iconv = require('iconv-lite');

request(options, function(error, response, body) {
  var decodedBody = iconv.decode(body, 'iso-8859-15');
  return callback(null, decodedBody)
 }
});

Great, now I have beautiful accents.

2) update this data, then send it back to the distant server using the first code block above.

Actually I used to do exactly the same operations with PHP, which worked fine : instead of sending the "raw" UTF-8 text message (postFields object), it was passed into the utf8_decode() func and the distant server displayed correctly the accents. I gave a try to js implementation of this function => the accents are correctly sent, but this time all other normal chars are turned weird...

Hoping this is clear enough, if you need more details, code, or practical examples : no problem! Thanks for reading :)

Upvotes: 0

Views: 1683

Answers (1)

Santiago G. Mar&#237;n
Santiago G. Mar&#237;n

Reputation: 95

Well, after 3 years and 10 months I faced a similar problem but I managed to find a solution!

To resolve this, you need to use urlencode and body instead of form, like this:

const urlencode = require('urlencode');
const request = require('request');

const fields = {
  message: 'é à ç'
};

const opts = {
  uri: 'https://localhost',

  headers: {
    'Content-type': 'application/x-www-form-urlencoded; charset=ISO-8859-1'
  },

  body: urlencode.stringify(fields, {
    charset: 'latin1'
  })
};

request.post(opts, (err, res, body) => {
  if (err) {
    throw err;
  }

  console.log(res.statusCode);
});

This is because request encodes form fields using UTF-8 so you never get Latin-1 characters even if you encode them before (you'll end up with re-encoded characters).

I've tested this and it works as expected, so I hope this helps someone else!

Upvotes: 1

Related Questions