CODEWITHSUNDEEP
javacharacter
black666
black666

Reputation: 3027

Remove "empty" character from String

I'm using a framwork which returns malformed Strings with "empty" characters from time to time.

"foobar" for example is represented by: [,f,o,o,b,a,r]

The first character is NOT a whitespace (' '), so a System.out.printlin() would return "foobar" and not " foobar". Yet, the length of the String is 7 instead of 6. Obviously this makes most String methods (equals, split, substring,..) useless. Is there a way to remove empty characters from a String?

I tried to build a new String like this:

StringBuilder sb = new StringBuilder();
for (final char character : malformedString.toCharArray()) {
  if (Character.isDefined(character)) {
    sb.append(character);
  }
}
sb.toString();

Unfortunately this doesn't work. Same with the following code:

StringBuilder sb = new StringBuilder();
for (final Character character : malformedString.toCharArray()) {
  if (character != null) {
    sb.append(character);
  }
}
sb.toString();

I also can't check for an empty character like this:

   if (character == ''){
     //
   }

Obviously there is something wrong with the String .. but I can't change the framework I'm using or wait for them to fix it (if it is a bug within their framework). I need to handle this String and sanatize it.

Any ideas?

Upvotes: 17

Views: 41662

Answers (10)

Denis Rybnikov
Denis Rybnikov

Reputation: 179

You can try replace:

s.replace("\u200B", "")

or

s.replace("\uFEFF", "")

Kotlin:

s.filter { it == '\u200B' }

Upvotes: 0

Lalji Gajera
Lalji Gajera

Reputation: 471

Simply malformedString.trim() will solve the issue.

Upvotes: -1

Steve Smith
Steve Smith

Reputation: 2271

This is what worked for me:-

    StringBuilder sb = new StringBuilder();
    for (char character : myString.toCharArray()) {
        int i = (int) character;
        if (i > 0 && i <= 256) {
            sb.append(character);
        }
    }  
    return sb.toString();

The int value of my NULL characters was in the region of 8103 or something.

Upvotes: 0

RightHandedMonkey
RightHandedMonkey

Reputation: 1728

A very simple way to remove the UTF-8 BOM from a string, using substring as Denis Tulskiy suggested. No looping needed. Just checks the first character for the mark and skips it if needed. 

public static String removeUTF8BOM(String s) {
    if (s.startsWith("\uFEFF")) {
        s = s.substring(1);
    }
    return s;
}

I needed to add this to my code when using the Apache HTTPClient EntityUtil to read from a webserver. The webserver was not sending the blank mark but it was getting pulled in while reading the input stream. Original article can be found here.

Upvotes: 7

Ilia Altshuler
Ilia Altshuler

Reputation: 1

for (int i = 0; i < s.length(); i++)
    if (s.charAt(i) == ' ') {
        your code....
    }

Upvotes: -1

BalusC
BalusC

Reputation: 1108537

It's probably the NULL character which is represented by \0. You can get rid of it by String#trim().

To nail down the exact codepoint, do so:

for (char c : string.toCharArray()) {
    System.out.printf("U+%04x ", (int) c);
}

Then you can find the exact character here.

Update: as per the update:

Anyone know of a way to just include a range of valid characters instead of excluding 95% of the UTF8 range?

You can do that with help of regex. See the answer of @polygenelubricants here and this answer.

On the other hand, you can also just fix the problem in its root instead of workarounding it. Either update the files to get rid of the BOM mark, it's a legacy way to distinguish UTF-8 files from others which is nowadays worthless, or use a Reader which recognizes and skips the BOM. Also see this question.

Upvotes: 17

polygenelubricants
polygenelubricants

Reputation: 383676

Regex would be an appropriate way to sanitize the string from unwanted Unicode characters in this case.

String sanitized = dirty.replaceAll("[\uFEFF-\uFFFF]", "");

This will replace all char in \uFEFF-\uFFFF range with the empty string.

The [...] construct is called a character class, e.g. [aeiou] matches one of any of the lowercase vowels, [^aeiou] matches anything but.

You can do one of these two approaches:

  • replaceAll("[blacklist]", "")
  • replaceAll("[^whitelist]", "")

References

Upvotes: 21

black666
black666

Reputation: 3027

Thank you Johannes Rössel. It actually was '\uFEFF'

The following code works:

 final StringBuilder sb = new StringBuilder();
    for (final char character : body.toCharArray()) {
       if (character != '\uFEFF') {
          sb.append(character);
       }
     }  
 final String sanitzedString = sb.toString();

Anyone know of a way to just include a range of valid characters instead of excluding 95% of the UTF8 range?

Upvotes: 2

ESP
ESP

Reputation: 13

trim left or right removes white spaces. does it has a colon before space?

even more: a=(long) string[0]; will show u the char code, and u can use replace() or substring.

Upvotes: 1

daiglebagel
daiglebagel

Reputation:

You could check for the whitespace like this:

if (character.equals(' ')){ // }

Upvotes: -3

Related Questions