CODEWITHSUNDEEP
javastringspringhttpurlconnection
EpicPandaForce
EpicPandaForce

Reputation: 81588

String content sent as byte[] gets its content escaped as query string

I need to send a String over the network, and I'm getting it on the other side in a REST Controller in Spring Boot.

I'm sending the data with the following lines:

byte[] content = sw.toString().getBytes(StandardCharsets.UTF_8); //sw is StringWriter
httpUrlConnection = (HttpURLConnection)new URL("http://127.0.0.1:8080/upload").openConnection();
httpUrlConnection.setDoInput(false);
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setFixedLengthStreamingMode(content.length);
httpUrlConnection.connect();
httpUrlConnection.getOutputStream().write(content);
httpUrlConnection.getOutputStream().flush();

And I'm receiving like this:

@RequestMapping(value="/upload", method = RequestMethod.POST)
public ResponseEntity<String> upload(@RequestBody byte[] uploadedData)
    throws NoSuchAlgorithmException, InvalidKeyException, IOException {
  if(uploadedData == null) {
    log.info("Uploaded data was null.");
    return new ResponseEntity<String>("Data was null after upload.", HttpStatus.INTERNAL_SERVER_ERROR);
  }

  String receivedData = new String(uploadedData, StandardCharsets.UTF_8);
  log.info("" + receivedData);

But while what I'm sending is this...

-----BEGIN CERTIFICATE REQUEST-----
MIIB0DCCATkCAQAwgY8xCzAJBgNVBAYTAkFVMSgwJgYDVQQKDB9UaGUgTGVnaW9u
IG9mIHRoZSBCb3VuY3kgQ2FzdGxlMRIwEAYDVQQHDAlNZWxib3VybmUxETAPBgNV
BAgMCFZpY3RvcmlhMS8wLQYJKoZIhvcNAQkBFiBmZWVkYmFjay1jcnlwdG9AYm91
bmN5Y2FzdGxlLm9yZzCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAqE+pEyKS
21JXgeldS83E+SSmQ4s2xYPqV7Yw2ebUKsxnbBz1KcXHdHS1MJ9PYzBJgogdigZW
6hEQ83edP/Ay/EQzGqeKUzFqNEsQh3PSdbF9N5k7b81tQHUbfIbNu1ofSBNa/Eit
MkOj1NAmwivpW0AA8aPZhGzYLYWcp0lsC78CAwEAAaAAMA0GCSqGSIb3DQEBBQUA
A4GBAB0p0ySmfMkm3z8H4P8WwWJ8bMO2RNXEx0i9fU2ncJfdY0zEPYvM6zpUhwJP
T9DsQBPdSy+VLbJ/PtYoiKIcupd+vriGYn3mqckXy7RBLqpiVsnw1rGE28oG4I9N
u0p2AwDuC+KNuHgtrGxYrRnFTRKZpj2AoGuW1a6eSaNOhPeq
-----END CERTIFICATE REQUEST-----

What I get is this:

-----BEGIN+CERTIFICATE+REQUEST-----%0D%0AMIIB0DCCATkCAQAwgY8xCzAJBgNVBAYTAkFVMSgwJgYDVQQKDB9UaGUgTGVnaW9u%0D%0AIG9mIHRoZSBCb3VuY3kgQ2FzdGxlMRIwEAYDVQQHDAlNZWxib3VybmUxETAPBgNV%0D%0ABAgMCFZpY3RvcmlhMS8wLQYJKoZIhvcNAQkBFiBmZWVkYmFjay1jcnlwdG9AYm91%0D%0AbmN5Y2FzdGxlLm9yZzCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAqE+pEyKS%0D%0A21JXgeldS83E+SSmQ4s2xYPqV7Yw2ebUKsxnbBz1KcXHdHS1MJ9PYzBJgogdigZW%0D%0A6hEQ83edP%2FAy%2FEQzGqeKUzFqNEsQh3PSdbF9N5k7b81tQHUbfIbNu1ofSBNa%2FEit%0D%0AMkOj1NAmwivpW0AA8aPZhGzYLYWcp0lsC78CAwEAAaAAMA0GCSqGSIb3DQEBBQUA%0D%0AA4GBAB0p0ySmfMkm3z8H4P8WwWJ8bMO2RNXEx0i9fU2ncJfdY0zEPYvM6zpUhwJP%0D%0AT9DsQBPdSy+VLbJ%2FPtYoiKIcupd+vriGYn3mqckXy7RBLqpiVsnw1rGE28oG4I9N%0D%0Au0p2AwDuC+KNuHgtrGxYrRnFTRKZpj2AoGuW1a6eSaNOhPeq%0D%0A-----END+CERTIFICATE+REQUEST-----%0D%0A=

This is not good because it kills the PEM parser.

However, I could overcome this problem by using the following transformations, and the String becomes parseable...

  receivedData = receivedData.replace("-----BEGIN+CERTIFICATE+REQUEST-----", "-----BEGIN CERTIFICATE REQUEST-----");
  receivedData = receivedData.replace("-----END+CERTIFICATE+REQUEST-----", "-----END CERTIFICATE REQUEST-----");
  receivedData = receivedData.replace("%0D%0A", "\r\n");
  receivedData = receivedData.replace("%2F", "/");
  receivedData = receivedData.substring(0, receivedData.length()-1);

In which case parsing was successful...

2014-10-18 16:26:43.864  INFO 2432 --- [nio-8080-exec-1] demo.HelloController                     : PemParser returned: org.spongycastle.pkcs.PKCS10CertificationRequest@f5c81fb9
2014-10-18 16:26:43.870  INFO 2432 --- [nio-8080-exec-1] demo.HelloController                     : SUCCESS

...but I don't think this is a really viable way of doing things as it doesn't look very stable. There has to be a better way. I tried URLEncoder.decode(receivedData, "UTF-8"); but it removes + characters from inside the actual data, which is also not good because it makes the data unparseable.

Does anyone have any useful ideas on how to prevent this URL escaping when I'm sending a String over the network?

Upvotes: 0

Views: 586

Answers (1)

EpicPandaForce
EpicPandaForce

Reputation: 81588

Well, I still don't know the answer for why that happened, but I did make a workaround.

Instead of just sending over the String byte[ ] content, I wrapped the data in a JSON Object with https://jsonp.java.net/index.html according to http://docs.oracle.com/javaee/7/tutorial/doc/jsonp003.htm.

JsonObject model = Json.createObjectBuilder().add("data", sw.toString()).build();

httpsUrlConnection = (HttpsURLConnection)new URL("http://127.0.0.1:8443/upload").openConnection();

httpsUrlConnection.setDoInput(true);
httpsUrlConnection.setDoOutput(true);
httpsUrlConnection.setRequestProperty("Content-Type", "application/json");
httpsUrlConnection.setRequestMethod("POST");
httpsUrlConnection.connect();
try(JsonWriter jsonWriter = Json.createWriter(httpsUrlConnection.getOutputStream()))
{
    jsonWriter.write(model);  
}
httpsUrlConnection.getOutputStream().flush();
System.out.println("Response code: " + httpsUrlConnection.getResponseCode());

Afterwards, on the Spring Boot side, I set it up so that I would receive the JSON object as per https://stackoverflow.com/a/8946142/2413303 :

public class DataObject {
  private String data;

  public String getData() {
    return data;
  }

  public void setData(String data) {
    this.data = data;
  }
}

And in Controller:

@RequestMapping(value="/upload", method = RequestMethod.POST, consumes="application/json")
  public ResponseEntity<String> upload(@RequestBody DataObject uploadedData)
    throws NoSuchAlgorithmException, InvalidKeyException, IOException {
    if(uploadedData == null) {
      log.info("Uploaded data was null.");
      return new ResponseEntity<String>("Data was null after upload.", HttpStatus.INTERNAL_SERVER_ERROR);
    }
    String receivedData = uploadedData.getData();
    log.info("" + receivedData);

This way the characters were not escaped:

2014-10-20 13:03:15.550  INFO 6924 --- [nio-8443-exec-3] demo.HelloController                     : -----BEGIN CERTIFICATE REQUEST-----

EDIT: I have been told that the problem was that with no content type set, Spring defaults to application/x-www-form-urlencoded, because it should have been application/octet-stream.

EDIT2: With octet-stream, sending it over escaped \r and \n to \\r and \\n, and added a " before and after the string for some reason. Not sure why.

Upvotes: 1

Related Questions