CODEWITHSUNDEEP
javaurlurlencode
Cheok Yan Cheng
Cheok Yan Cheng

Reputation: 42710

URLEncoder not able to translate space character

I am expecting

System.out.println(java.net.URLEncoder.encode("Hello World", "UTF-8"));

to output:

Hello%20World

(20 is ASCII Hex code for space)

However, what I get is:

Hello+World

Am I using the wrong method? What is the correct method I should be using?

Upvotes: 260

Views: 252150

Answers (19)

MrTux
MrTux

Reputation: 34003

If you want to encode URI path components, you can also use standard JDK functions, e.g.

public static String encodeURLPathComponent(String path) {
    try {
        return new URI(null, null, path, null).toASCIIString();
    } catch (URISyntaxException e) {
        // do some error handling
    }
    return "";
}

The URI class can also be used to encode different parts of or whole URIs.

Update: I just realized that this does not work if there is a colon before a slash in the path or when the part before the colon is not valid URI scheme. Also, it normalizes all unicode to NFC.

Upvotes: 8

rjdkolb
rjdkolb

Reputation: 11858

I was already using Feign so UriUtils was available to me but Spring UrlUtils was not.

<!-- https://mvnrepository.com/artifact/io.github.openfeign/feign-core -->
<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-core</artifactId>
    <version>11.8</version>
</dependency>

My Feign test code:

import feign.template.UriUtils;

System.out.println(UriUtils.encode("Hello World"));

Outputs:

Hello%20World

As the class suggests, it encodes URIs and not URLs but the OP asked about URIs and not URLs.

System.out.println(UriUtils.encode("https://some-host.net/dav/files/selling_Rosetta Stone Case Study.png.aes"));

Outputs:

https%3A%2F%2Fsome-host.net%2Fdav%2Ffiles%2Fselling_Rosetta%20Stone%20Case%20Study.png.aes

Upvotes: 2

Chandan Kumar
Chandan Kumar

Reputation: 21

Try below approach:

Add a new dependency

<!-- https://mvnrepository.com/artifact/org.apache.tomcat/tomcat-catalina -->
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-catalina</artifactId>
    <version>10.0.13</version>
</dependency>

Now do as follows:

String str = "Hello+World"; // For "Hello World", decoder is not required
// import java.net.URLDecoder;
String newURL = URLDecoder.decode(str, StandardCharsets.UTF_8);
// import org.apache.catalina.util.URLEncoder;
System.out.println(URLEncoder.DEFAULT.encode(newURL, StandardCharsets.UTF_8));

You'll get the output as:

Hello%20World

Upvotes: 0

tchudyk
tchudyk

Reputation: 614

It's not one-liner, but you can use:

URL url = new URL("https://some-host.net/dav/files/selling_Rosetta Stone Case Study.png.aes");
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
System.out.println(uri.toString());

This will give you an output:

https://some-host.net/dav/files/selling_Rosetta%20Stone%20Case%20Study.png.aes

Upvotes: 5

Daniel
Daniel

Reputation: 28084

"+" is correct. If you really need %20, then replace the Plusses yourself afterwards.

Warning: This answer is heavily disputed (+8 vs. -6), so take this with a grain of salt.

Upvotes: -1

gourab ghosh
gourab ghosh

Reputation: 169

If you are using jetty then org.eclipse.jetty.util.URIUtil will solve the issue.

String encoded_string = URIUtil.encodePath(not_encoded_string).toString();

Upvotes: 5

LeO
LeO

Reputation: 5238

Although quite old, nevertheless a quick response:

Spring provides UriUtils - with this you can specify how to encoded and which part is it related from an URI, e.g.

encodePathSegment
encodePort
encodeFragment
encodeUriVariables
....

I use them cause we already using Spring, i.e. no additonal library is required!

Upvotes: 6

Benny Bottema
Benny Bottema

Reputation: 11493

The other answers either present a manual string replacement, URLEncoder which actually encodes for HTML format, Apache's abandoned URIUtil, or using Guava's UrlEscapers. The last one is fine, except it doesn't provide a decoder.

Apache Commons Lang provides the URLCodec, which encodes and decodes according to URL format rfc3986.

String encoded = new URLCodec().encode(str);
String decoded = new URLCodec().decode(str);

If you are already using Spring, you can also opt to use its UriUtils class as well.

Upvotes: 5

pyb
pyb

Reputation: 5269

A space is encoded to %20 in URLs, and to + in forms submitted data (content type application/x-www-form-urlencoded). You need the former.

Using Guava:

dependencies {
     compile 'com.google.guava:guava:23.0'
     // or, for Android:
     compile 'com.google.guava:guava:23.0-android'
}

You can use UrlEscapers:

String encodedString = UrlEscapers.urlFragmentEscaper().escape(inputString);

Don't use String.replace, this would only encode the space. Use a library instead.

Upvotes: 80

Chrispix
Chrispix

Reputation: 18231

Just been struggling with this too on Android, managed to stumble upon Uri.encode(String, String) while specific to android (android.net.Uri) might be useful to some.

static String encode(String s, String allow)

https://developer.android.com/reference/android/net/Uri.html#encode(java.lang.String, java.lang.String)

Upvotes: 6

Pregunton
Pregunton

Reputation: 87

Am I using the wrong method? What is the correct method I should be using?

Yes, this method java.net.URLEncoder.encode wasn't made for converting " " to "20%" according to spec (source).

The space character " " is converted into a plus sign "+".

Even this is not the correct method, you can modify this to: System.out.println(java.net.URLEncoder.encode("Hello World", "UTF-8").replaceAll("\\+", "%20"));have a nice day =).

Upvotes: -1

Hitesh Kumar
Hitesh Kumar

Reputation: 653

This worked for me

org.apache.catalina.util.URLEncoder ul = new org.apache.catalina.util.URLEncoder().encode("MY URL");

Upvotes: 3

dogbane
dogbane

Reputation: 274612

This behaves as expected. The URLEncoder implements the HTML Specifications for how to encode URLs in HTML forms.

From the javadocs:

This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format.

and from the HTML Specification:

application/x-www-form-urlencoded

Forms submitted with this content type must be encoded as follows:

  1. Control names and values are escaped. Space characters are replaced by `+'

You will have to replace it, e.g.:

System.out.println(java.net.URLEncoder.encode("Hello World", "UTF-8").replace("+", "%20"));

Upvotes: 261

IloveIniesta
IloveIniesta

Reputation: 350

USE MyUrlEncode.URLencoding(String url , String enc) to handle the problem

    public class MyUrlEncode {
    static BitSet dontNeedEncoding = null;
    static final int caseDiff = ('a' - 'A');
    static {
        dontNeedEncoding = new BitSet(256);
        int i;
        for (i = 'a'; i <= 'z'; i++) {
            dontNeedEncoding.set(i);
        }
        for (i = 'A'; i <= 'Z'; i++) {
            dontNeedEncoding.set(i);
        }
        for (i = '0'; i <= '9'; i++) {
            dontNeedEncoding.set(i);
        }
        dontNeedEncoding.set('-');
        dontNeedEncoding.set('_');
        dontNeedEncoding.set('.');
        dontNeedEncoding.set('*');
        dontNeedEncoding.set('&');
        dontNeedEncoding.set('=');
    }
    public static String char2Unicode(char c) {
        if(dontNeedEncoding.get(c)) {
            return String.valueOf(c);
        }
        StringBuffer resultBuffer = new StringBuffer();
        resultBuffer.append("%");
        char ch = Character.forDigit((c >> 4) & 0xF, 16);
            if (Character.isLetter(ch)) {
            ch -= caseDiff;
        }
        resultBuffer.append(ch);
            ch = Character.forDigit(c & 0xF, 16);
            if (Character.isLetter(ch)) {
            ch -= caseDiff;
        }
         resultBuffer.append(ch);
        return resultBuffer.toString();
    }
    private static String URLEncoding(String url,String enc) throws UnsupportedEncodingException {
        StringBuffer stringBuffer = new StringBuffer();
        if(!dontNeedEncoding.get('/')) {
            dontNeedEncoding.set('/');
        }
        if(!dontNeedEncoding.get(':')) {
            dontNeedEncoding.set(':');
        }
        byte [] buff = url.getBytes(enc);
        for (int i = 0; i < buff.length; i++) {
            stringBuffer.append(char2Unicode((char)buff[i]));
        }
        return stringBuffer.toString();
    }
    private static String URIEncoding(String uri , String enc) throws UnsupportedEncodingException { //对请求参数进行编码
        StringBuffer stringBuffer = new StringBuffer();
        if(dontNeedEncoding.get('/')) {
            dontNeedEncoding.clear('/');
        }
        if(dontNeedEncoding.get(':')) {
            dontNeedEncoding.clear(':');
        }
        byte [] buff = uri.getBytes(enc);
        for (int i = 0; i < buff.length; i++) {
            stringBuffer.append(char2Unicode((char)buff[i]));
        }
        return stringBuffer.toString();
    }

    public static String URLencoding(String url , String enc) throws UnsupportedEncodingException {
        int index = url.indexOf('?');
        StringBuffer result = new StringBuffer();
        if(index == -1) {
            result.append(URLEncoding(url, enc));
        }else {
            result.append(URLEncoding(url.substring(0 , index),enc));
            result.append("?");
            result.append(URIEncoding(url.substring(index+1),enc));
        }
        return result.toString();
    }

}

Upvotes: -2

Akhil Sikri
Akhil Sikri

Reputation: 9

use character-set "ISO-8859-1" for URLEncoder

Upvotes: -8

McDowell
McDowell

Reputation: 108899

Hello+World is how a browser will encode form data (application/x-www-form-urlencoded) for a GET request and this is the generally accepted form for the query part of a URI.

http://host/path/?message=Hello+World

If you sent this request to a Java servlet, the servlet would correctly decode the parameter value. Usually the only time there are issues here is if the encoding doesn't match.

Strictly speaking, there is no requirement in the HTTP or URI specs that the query part to be encoded using application/x-www-form-urlencoded key-value pairs; the query part just needs to be in the form the web server accepts. In practice, this is unlikely to be an issue.

It would generally be incorrect to use this encoding for other parts of the URI (the path for example). In that case, you should use the encoding scheme as described in RFC 3986.

http://host/Hello%20World

More here.

Upvotes: 12

Fredrik Widerberg
Fredrik Widerberg

Reputation: 3108

Check out the java.net.URI class.

Upvotes: -1

axtavt
axtavt

Reputation: 242696

This class perform application/x-www-form-urlencoded-type encoding rather than percent encoding, therefore replacing with + is a correct behaviour.

From javadoc:

When encoding a String, the following rules apply:

  • The alphanumeric characters "a" through "z", "A" through "Z" and "0" through "9" remain the same.
  • The special characters ".", "-", "*", and "_" remain the same.
  • The space character " " is converted into a plus sign "+".
  • All other characters are unsafe and are first converted into one or more bytes using some encoding scheme. Then each byte is represented by the 3-character string "%xy", where xy is the two-digit hexadecimal representation of the byte. The recommended encoding scheme to use is UTF-8. However, for compatibility reasons, if an encoding is not specified, then the default encoding of the platform is used.

Upvotes: 30

fmucar
fmucar

Reputation: 14558

Encode Query params

org.apache.commons.httpclient.util.URIUtil
    URIUtil.encodeQuery(input);

OR if you want to escape chars within URI

public static String escapeURIPathParam(String input) {
  StringBuilder resultStr = new StringBuilder();
  for (char ch : input.toCharArray()) {
   if (isUnsafe(ch)) {
    resultStr.append('%');
    resultStr.append(toHex(ch / 16));
    resultStr.append(toHex(ch % 16));
   } else{
    resultStr.append(ch);
   }
  }
  return resultStr.toString();
 }

 private static char toHex(int ch) {
  return (char) (ch < 10 ? '0' + ch : 'A' + ch - 10);
 }

 private static boolean isUnsafe(char ch) {
  if (ch > 128 || ch < 0)
   return true;
  return " %$&+,/:;=?@<>#%".indexOf(ch) >= 0;
 }

Upvotes: 29

Related Questions