CODEWITHSUNDEEP
javaxmlencoding
John Goering
John Goering

Reputation: 39040

Best way to encode text data for XML in Java?

Very similar to this question, except for Java.

What is the recommended way of encoding strings for an XML output in Java. The strings might contain characters like "&", "<", etc.

Upvotes: 106

Views: 223055

Answers (21)

Mike B
Mike B

Reputation: 1669

This question is eight years old and still not a fully correct answer! No, you should not have to import an entire third party API to do this simple task. Bad advice.

The following method will:

  • correctly handle characters outside the basic multilingual plane
  • escape characters required in XML
  • escape any non-ASCII characters, which is optional but common
  • replace illegal characters in XML 1.0 with the Unicode substitution character. There is no best option here - removing them is just as valid.

I've tried to optimise for the most common case, while still ensuring you could pipe /dev/random through this and get a valid string in XML.

public static String encodeXML(String s) {
    StringBuilder sb = new StringBuilder();
    int len = s.length();
    for (int i=0;i<len;) {
        int c = s.codePointAt(i);
        if (c < 0x80) {      // ASCII range: test most common case first
            if (c < 0x20 && (c != '\t' && c != '\r' && c != '\n')) {
                // Illegal XML character, even encoded. Skip or substitute
                sb.append("&#xfffd;");   // Unicode replacement character
            } else {
                switch(c) {
                  case '&':  sb.append("&amp;"); break;
                  case '>':  sb.append("&gt;"); break;
                  case '<':  sb.append("&lt;"); break;
                  // Uncomment next two if encoding for an XML attribute
//                  case '\''  sb.append("&apos;"); break;
//                  case '\"'  sb.append("&quot;"); break;
                  // Uncomment next three if you prefer, but not required
//                  case '\n'  sb.append("&#10;"); break;
//                  case '\r'  sb.append("&#13;"); break;
//                  case '\t'  sb.append("&#9;"); break;

                  default:   sb.append((char)c);
                }
            }
        } else if ((c >= 0xd800 && c <= 0xdfff) || c == 0xfffe || c == 0xffff) {
            // Illegal XML character, even encoded. Skip or substitute
            sb.append("&#xfffd;");   // Unicode replacement character
        } else {
            sb.append("&#x");
            sb.append(Integer.toHexString(c));
            sb.append(';');
        }
        i += c <= 0xffff ? 1 : 2;
    }
    return sb.toString();
}

Edit: for those who continue to insist it foolish to write your own code for this when there are perfectly good Java APIs to deal with XML, you might like to know that the StAX API included with Oracle Java 8 (I haven't tested others) fails to encode CDATA content correctly: it doesn't escape ]]> sequences in the content. A third party library, even one that's part of the Java core, is not always the best option.

Upvotes: 19

Dharmendrasinh Chudasama
Dharmendrasinh Chudasama

Reputation: 1117

I have created my wrapper here, hope it will helps a lot, Click here You can modify depends on your requirements

Upvotes: -1

chaotic3quilibrium
chaotic3quilibrium

Reputation: 5924

While I agree with Jon Skeet in principle, sometimes I don't have the option to use an external XML library. And I find it peculiar the two functions to escape/unescape a simple value (attribute or tag, not full document) are not available in the standard XML libraries included with Java.

As a result and based on the different answers I have seen posted here and elsewhere, here is the solution I've ended up creating (nothing worked as a simple copy/paste):

  public final static String ESCAPE_CHARS = "<>&\"\'";
  public final static List<String> ESCAPE_STRINGS = Collections.unmodifiableList(Arrays.asList(new String[] {
      "&lt;"
    , "&gt;"
    , "&amp;"
    , "&quot;"
    , "&apos;"
  }));

  private static String UNICODE_NULL = "" + ((char)0x00); //null
  private static String UNICODE_LOW =  "" + ((char)0x20); //space
  private static String UNICODE_HIGH = "" + ((char)0x7f);

  //should only be used for the content of an attribute or tag      
  public static String toEscaped(String content) {
    String result = content;
    
    if ((content != null) && (content.length() > 0)) {
      boolean modified = false;
      StringBuilder stringBuilder = new StringBuilder(content.length());
      for (int i = 0, count = content.length(); i < count; ++i) {
        String character = content.substring(i, i + 1);
        int pos = ESCAPE_CHARS.indexOf(character);
        if (pos > -1) {
          stringBuilder.append(ESCAPE_STRINGS.get(pos));
          modified = true;
        }
        else {
          if (    (character.compareTo(UNICODE_LOW) > -1)
               && (character.compareTo(UNICODE_HIGH) < 1)
             ) {
            stringBuilder.append(character);
          }
          else {
            //Per URL reference below, Unicode null character is always restricted from XML
            //URL: https://en.wikipedia.org/wiki/Valid_characters_in_XML
            if (character.compareTo(UNICODE_NULL) != 0) {
              stringBuilder.append("&#" + ((int)character.charAt(0)) + ";");
            }
            modified = true;
          }
        }
      }
      if (modified) {
        result = stringBuilder.toString();
      }
    }
    
    return result;
  }

The above accommodates several different things:

  1. avoids using char based logic until it absolutely has to - improves unicode compatibility
  2. attempts to be as efficient as possible given the probability is the second "if" condition is likely the most used pathway
  3. is a pure function; i.e. is thread-safe
  4. optimizes nicely with the garbage collector by only returning the contents of the StringBuilder if something actually changed - otherwise, the original string is returned

At some point, I will write the inversion of this function, toUnescaped(). I just don't have time to do that today. When I do, I will come update this answer with the code. :)

Upvotes: 6

wizston
wizston

Reputation: 41

Here's what I found after searching everywhere looking for a solution:

Get the Jsoup library:

<!-- https://mvnrepository.com/artifact/org.jsoup/jsoup -->
<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.12.1</version>
</dependency>

Then:

import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Entities
import org.jsoup.parser.Parser

String xml = '''<?xml version = "1.0"?>
<SOAP-ENV:Envelope
   xmlns:SOAP-ENV = "http://www.w3.org/2001/12/soap-envelope"
   SOAP-ENV:encodingStyle = "http://www.w3.org/2001/12/soap-encoding">

   <SOAP-ENV:Body xmlns:m = "http://www.example.org/quotations">
      <m:GetQuotation>
         <m:QuotationsName> MiscroSoft@G>>gle.com </m:QuotationsName>
      </m:GetQuotation>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>'''



Document doc = Jsoup.parse(new ByteArrayInputStream(xml.getBytes("UTF-8")), "UTF-8", "", Parser.xmlParser())
doc.outputSettings().charset("UTF-8")
doc.outputSettings().escapeMode(Entities.EscapeMode.base)

println doc.toString()

Hope this helps someone

Upvotes: 0

jschnasse
jschnasse

Reputation: 9558

If you are looking for a library to get the job done, try:

  1. Guava 26.0 documented here

    return XmlEscapers.xmlContentEscaper().escape(text);

    Note: There is also an xmlAttributeEscaper()

  2. Apache Commons Text 1.4 documented here

    StringEscapeUtils.escapeXml11(text)

    Note: There is also an escapeXml10() method

Upvotes: 4

raman rayat
raman rayat

Reputation: 414

Just replace

 & with &amp;

And for other characters: 

> with &gt;
< with &lt;
\" with &quot;
' with &apos;

Upvotes: 0

Software Craftsman
Software Craftsman

Reputation: 3269

You could use the Enterprise Security API (ESAPI) library, which provides methods like encodeForXML and encodeForXMLAttribute. Take a look at the documentation of the Encoder interface; it also contains examples of how to create an instance of DefaultEncoder.

Upvotes: 2

Fabian Steeg
Fabian Steeg

Reputation: 45744

As others have mentioned, using an XML library is the easiest way. If you do want to escape yourself, you could look into StringEscapeUtils from the Apache Commons Lang library.

Upvotes: 130

Dariusz
Dariusz

Reputation: 22271

For those looking for the quickest-to-write solution: use methods from apache commons-lang:

Remember to include dependency:

<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-lang3</artifactId>
  <version>3.5</version> <!--check current version! -->
</dependency>

Upvotes: 8

iCrazybest
iCrazybest

Reputation: 3115

public String escapeXml(String s) {
    return s.replaceAll("&", "&amp;").replaceAll(">", "&gt;").replaceAll("<", "&lt;").replaceAll("\"", "&quot;").replaceAll("'", "&apos;");
}

Upvotes: 9

K Victor Rajan
K Victor Rajan

Reputation: 1

Try to encode the XML using Apache XML serializer 

//Serialize DOM
OutputFormat format    = new OutputFormat (doc); 
// as a String
StringWriter stringOut = new StringWriter ();    
XMLSerializer serial   = new XMLSerializer (stringOut, 
                                          format);
serial.serialize(doc);
// Display the XML
System.out.println(stringOut.toString());

Upvotes: 0

Mike
Mike

Reputation: 1400

Here's an easy solution and it's great for encoding accented characters too!

String in = "Hi Lârry & Môe!";

StringBuilder out = new StringBuilder();
for(int i = 0; i < in.length(); i++) {
    char c = in.charAt(i);
    if(c < 31 || c > 126 || "<>\"'\\&".indexOf(c) >= 0) {
        out.append("&#" + (int) c + ";");
    } else {
        out.append(c);
    }
}

System.out.printf("%s%n", out);

Outputs

Hi L&#226;rry &#38; M&#244;e!

Upvotes: 1

Pointer Null
Pointer Null

Reputation: 40380

Try this:

String xmlEscapeText(String t) {
   StringBuilder sb = new StringBuilder();
   for(int i = 0; i < t.length(); i++){
      char c = t.charAt(i);
      switch(c){
      case '<': sb.append("&lt;"); break;
      case '>': sb.append("&gt;"); break;
      case '\"': sb.append("&quot;"); break;
      case '&': sb.append("&amp;"); break;
      case '\'': sb.append("&apos;"); break;
      default:
         if(c>0x7e) {
            sb.append("&#"+((int)c)+";");
         }else
            sb.append(c);
      }
   }
   return sb.toString();
}

Upvotes: 13

Steve Mitchell
Steve Mitchell

Reputation: 2070

StringEscapeUtils.escapeXml() does not escape control characters (< 0x20). XML 1.1 allows control characters; XML 1.0 does not. For example, XStream.toXML() will happily serialize a Java object's control characters into XML, which an XML 1.0 parser will reject.

To escape control characters with Apache commons-lang, use

NumericEntityEscaper.below(0x20).translate(StringEscapeUtils.escapeXml(str))

Upvotes: 9

Jasper Krijgsman
Jasper Krijgsman

Reputation: 1028

The behavior of StringEscapeUtils.escapeXml() has changed from Commons Lang 2.5 to 3.0. It now no longer escapes Unicode characters greater than 0x7f.

This is a good thing, the old method was to be a bit to eager to escape entities that could just be inserted into a utf8 document.

The new escapers to be included in Google Guava 11.0 also seem promising: http://code.google.com/p/guava-libraries/issues/detail?id=799

Upvotes: 6

Greg Burdett
Greg Burdett

Reputation: 191

To escape XML characters, the easiest way is to use the Apache Commons Lang project, JAR downloadable from: http://commons.apache.org/lang/

The class is this: org.apache.commons.lang3.StringEscapeUtils;

It has a method named "escapeXml", that will return an appropriately escaped String.

Upvotes: 3

Amr Mostafa
Amr Mostafa

Reputation: 23987

While idealism says use an XML library, IMHO if you have a basic idea of XML then common sense and performance says template it all the way. It's arguably more readable too. Though using the escaping routines of a library is probably a good idea.

Consider this: XML was meant to be written by humans.

Use libraries for generating XML when having your XML as an "object" better models your problem. For example, if pluggable modules participate in the process of building this XML.

Edit: as for how to actually escape XML in templates, use of CDATA or escapeXml(string) from JSTL are two good solutions, escapeXml(string) can be used like this:

<%@taglib prefix="fn" uri="http://java.sun.com/jsp/jstl/functions"%>

<item>${fn:escapeXml(value)}</item>

Upvotes: 6

Aaron Digulla
Aaron Digulla

Reputation: 328724

Note: Your question is about escaping, not encoding. Escaping is using <, etc. to allow the parser to distinguish between "this is an XML command" and "this is some text". Encoding is the stuff you specify in the XML header (UTF-8, ISO-8859-1, etc).

First of all, like everyone else said, use an XML library. XML looks simple but the encoding+escaping stuff is dark voodoo (which you'll notice as soon as you encounter umlauts and Japanese and other weird stuff like "full width digits" (&#FF11; is 1)). Keeping XML human readable is a Sisyphus' task.

I suggest never to try to be clever about text encoding and escaping in XML. But don't let that stop you from trying; just remember when it bites you (and it will).

That said, if you use only UTF-8, to make things more readable you can consider this strategy:

  • If the text does contain '<', '>' or '&', wrap it in <![CDATA[ ... ]]>
  • If the text doesn't contain these three characters, don't warp it.

I'm using this in an SQL editor and it allows the developers to cut&paste SQL from a third party SQL tool into the XML without worrying about escaping. This works because the SQL can't contain umlauts in our case, so I'm safe.

Upvotes: 5

ng.
ng.

Reputation: 7189

Just use.

<![CDATA[ your text here ]]>

This will allow any characters except the ending 

]]>

So you can include characters that would be illegal such as & and >. For example.

<element><![CDATA[ characters such as & and > are allowed ]]></element>

However, attributes will need to be escaped as CDATA blocks can not be used for them.

Upvotes: 21

Fernando Migu&#233;lez
Fernando Migu&#233;lez

Reputation: 11316

Use JAXP and forget about text handling it will be done for you automatically.

Upvotes: 1

Jon Skeet
Jon Skeet

Reputation: 1502106

Very simply: use an XML library. That way it will actually be right instead of requiring detailed knowledge of bits of the XML spec.

Upvotes: 35

Related Questions