CODEWITHSUNDEEP
javaunicodejava.util.scanner
chigley
chigley

Reputation: 2592

Java Unicode Problems (I think)

I'm new to Java, so bear with me if I say anything stupid! I'm having a few problems, which I think are Unicode-related.

I'm using Scanner to read in tokenised commands from a text file, saved with UTF-8 encoding. Basically I want to first check that the command isn't equal to either "command1" or "command2" (I do something else in these cases), then otherwise read in a character. If the token isn't a single character, I'm going to output an error.

Here's my code:

public static void main(String[] args) throws FileNotFoundException {
    Scanner scanner = new Scanner(new File(args[0]));
    while (scanner.hasNext()) {
        String command = scanner.next();
        if (command.equals("command1")) {
            System.out.println("command: command1");
            // do something
        } else if (command.equals("command2")) {
            System.out.println("command: command2");
            // do something
        } else {
            if (command.length() == 1) {
                char c = command.charAt(0);
                System.out.println("character: " + c);
                // do something with c
            } else {
                System.err.println("error (string was " + command
                        + " with length " + command.length() + ")");
            }
        }
    }
}

And the contents of the text file whose filename I'm passing in args[0] for testing:

command1
x
y
command2
z
└
command1
╒
═

Expected output is:

command: command1
character: x
character: y
command: command2
character: z
character: └
command: command1
character: ╒
character:  ═

Actual output is:

command: command1
character: x
character: y
command: command2
character: z
error (string was └ with length 3)
command: command1
error (string was ╒ with length 3)
error (string was ═ with length 3)

As you can see, the non-standard characters are being seen as a 3-character string by Java. Strangely, if I copy/paste the one of the characters from the terminal output into a System.out.println("└".length()) statement, it correctly prints 1.

Any ideas on where I'm going wrong?
Thanks

Upvotes: 1

Views: 4326

Answers (2)

Olivier Croisier
Olivier Croisier

Reputation: 6149

I suspect your problem comes indeed from an encoding mismatch. Have you tried passing a Charset in the Scanner's constructor ?

Your code works perfectly on my system (Arch Linux 64b, java 6.0.30), with a default locale in UTF-8. If you run Windows, your locale may be Win-CP1252, which could be used by the Scanner.

Upvotes: 0

Calum
Calum

Reputation: 5926

When you open files in Java, the encoding (if you don't specify one) is taken from the file.encoding system property. This is almost never set to something that you want (if you're like me, you always want UTF-8).

To fix, explicitly specify your character set when you create your Scanner:

Scanner scanner = new Scanner(new File(args[0]), "UTF-8");

Upvotes: 9

Related Questions