nextLine() and while loop misbehavior in unicode-16

Question

Edit: This actually turned into quite an interesting problem. After some help from commenters, I posted a self-answer. I should mention that my project is in Unicode-16, which looks like it was the source of the trouble.

The problem is that the loop was not exiting as expected, in what appears to be trivially simple code:

import java.util.Scanner;

public class Lambda2 {
    public static void main(String[] args){
        Scanner in = new Scanner(System.in);
        String input = in.nextLine();
        while (!input.equals("exit")){
            System.out.println("input is \""+ input + "\"");
            System.out.println(input.equals("exit"));

            input = in.nextLine();
        }
        System.out.println("Thank you!");
    }
}

Run 1:

exit
Thank you!

So far, so good. But when I enter the loop, I run into trouble:

Run 2:

asdf
input is "asdf"
false
exit
input is "exit"
false
exit
input is "exit"
false

Last I checked "exit".equals("exit") should return true, not false. I've tried using trim() on my inputs in case there was some skullduggery with new lines... What in the world am I missing??

Answer 1

Neither of them posted an answer, but with the help of GBlodgett and StephenC, an interesting answer did eventually emerge.

The problem was that the project is, by necessity, in a UTF encoding, and a BOM character (U-FEFF) was being added to the beginning of the user input, making it 5 characters long.

The solution was to remove the BOM character immediately after collection:

input = input.replace("\uFEFF", "");

What is still somewhat mysterious, however, is why no BOM was added to the first input, but only to the subsequent ones. It seems like Run 1 should not have worked.

Answer 2

I tried this code and it is working fine.