CODEWITHSUNDEEP
javaencodingutf-8character-encodingutf
garryS
garryS

Reputation: 13

Java convert hex UTF8 to UTF-16

I am being sent some transactions data (in json format) from mainframe by a data capture product. In the JSON, they are supposed to send a value which is mainframe timeOfDay (TOD) value indicating a transaction timestamp equivalent. Instead of sending a value "stck":"00d2fde04305b72642"

they are instead sending me "stck":"\u00c3\u0092\u00c3\u00bd\u00c2\u00aa\u00c2\u009e\u001c\u00c2\u0089\u001cG"

when I asked them why they said

"the above ("stck":"00d2fde04305b72642") is binary data UTF-16 format. JSON doesn't work well with binary data so we have converted that to hex UTF-8 format. And that you can convert it back to UTF-16 on your side"

I need help doing this in java. I have seen multiple questions but nothing that does exactly what I am trying to do that is convert hex UTF-8 to UTF-16 which should hopefully look like "00d2fde04305b72642"

I have been able to find a question that shows how to convert this resulting TOD value ("stck":"00d2fde04305b72642") into transaction timestamp using java so I am covered on that part.

Upvotes: 0

Views: 1288

Answers (1)

erickson
erickson

Reputation: 269667

"They" are doing it wrong. They should simply encode the numeric value in base-16 and use the resulting string.

What they are doing instead is treating the bytes of the numeric value as characters, and encoding them with UTF-8. Then they take those bytes and apply a Unicode escape sequence on non-ASCII characters. Adding insult to injury, they responded with nonsense when asked about the process. It's a disaster on many levels.

The following should allow you to recover the data and convert to a Java timestamp.

import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.temporal.ChronoUnit;

public class SO45704851
{

  public static void main(String... argv)
    throws Exception
  {
    String example = "\u00c3\u0092\u00c3\u00bd\u00c2\u00aa\u00c2\u009e\u001c\u00c2\u0089\u001cG";
    long tod = sbrogliare(example);
    System.out.printf("ToD: 0x%016x%n", tod);
    Instant time = toInstant(tod);
    System.out.printf("Instant: %s%n", time);
  }

  /**
   * Clean up an infernal mess, miraculously bestowing a 64-bit time-of-day.
   */
  public static long sbrogliare(String garbage)
  {
    byte[] utf8 = new byte[garbage.length()];
    for (int idx = 0; idx < garbage.length(); ++idx)
      utf8[idx] = (byte) garbage.charAt(idx);
    String raw = new String(utf8, StandardCharsets.UTF_8);
    if (raw.length() != 8)
      throw new IllegalArgumentException();
    long n = 0;
    for (int idx = 0; idx < raw.length(); ++idx) {
      char ch = raw.charAt(idx);
      if (ch > 255)
        throw new IllegalArgumentException();
      n = n << 8 | ch;
    }
    return n;
  }

  private static final OffsetDateTime zero = OffsetDateTime.parse("1900-01-01T00:00Z");

  /**
   * Convert 64-bit time-of-day to {@code Instant}.
   */
  public static Instant toInstant(long tod)
  {
    long nanos = (125 * (tod >>> 32) << 23) + (125 * (tod & 0xFFFFFFFFL) >>> 9);
    return zero.plus(nanos, ChronoUnit.NANOS).toInstant();
  }

}

Upvotes: 1

Related Questions