CODEWITHSUNDEEP
clinuxunixunicodecharacter-encoding
james navarro
james navarro

Reputation: 1

Append text to an existing UTF16LE file

How can I write to an existing file with UTF16LE encoding? I've already used fopen(file, "a"); but the resulting file will be like this:

<?xml version="1.0" encoding="UTF-16" standalone="yes"?> 㰼㱤㱯㱣㰾㰊㰼㱰㱡㱧㱥㰠㱮㱡㱭㱥㰽㰢㱎㱏㱒㱍㱁㱌㰢㰾㰊㰼㱦㱩㱥㱬㱤㰠㱮㱡㱭㱥㰽㰢㱉㱤㱥㱮㱴㱩㱦㱩㱣㱡㱴㱩㱯㱮㸢㱔㱃㰳㰶㰰㰴㰰㰱㰭㰭㰭㰭㰱㰲㰷㰼㰯㱦㱩㱥㱬㱤㰾㰊㰼㱦㱩㱥㱬㱤㰠㱮㱡㱭㱥㰽㰢㱔㱲㱡㱣㱥㱡㱢㱩㱬㱩㱴㱹㸢㰱㰳㱖㱖㱖㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰭㰰㰰㰼㰯㱦㱩㱥㱬㱤㰾㰊㰼㱦㱩㱥㱬㱤㰠㱮㱡㱭㱥㰽㰢㱄㱥㱳㱣㱲㱩㱰㱴㱩㱯㱮㸢㱄㱥㱳㱣㱲㱩㱰㱴㱩㱯㱮㰀㰼㰯㱦㱩㱥㱬㱤㰾㰊㰼㰯㱰㱡㱧㱥㰾㰊㰼㰯㱤㱯㱣㰾㰊

I don't know how I can append a 2-byte character to this file.

Upvotes: 0

Views: 267

Answers (1)

Mike Kinghan
Mike Kinghan

Reputation: 61337

A UTF-16 character is not necessarily 2 bytes wide. It may be 2 bytes or 4 bytes (read up here).

The weird output you have posted most likely result from appending wchar_ts directly to the file, generating UTF-16 characters with a byte order that is the reverse of the right one, and these UTF-16 characters lie up in the "oriental" heights of the UTF-16 range.

Assuming from your question's tags that you are working with GCC on Linux, you may use the iconv library by including <inconv.h> to import a character-encoding conversion api. Here is a specimen program that converts the wchar_t array:

L'A',L'P',L'P',L'E',L'N',L'D',L'A',L'G',L'E' // "APPENDAGE"

to UTF-16LE and appends the result to the file "tdata.txt". It hard-codes a limit of 64 bytes on the converted length of output.

#include <stdio.h>
#include <stdlib.h>
#include <iconv.h>
#include <assert.h>

#define MAXOUT 64

int main(void)
{
    wchar_t appendage [] = {
        L'A',L'P',L'P',L'E',L'N',L'D',L'A',L'G',L'E'
    };
    wchar_t * inp = appendage; 
    char converted[MAXOUT];
    char * outp = converted;
    size_t remain_in = sizeof(appendage);
    size_t remain_out = MAXOUT;
    size_t conversions;
    size_t written;
    char const *tfile = "../tdata.txt";
    // Create the right converter from wchar_t to UTF-16LE
    iconv_t iconvdesc = iconv_open("UTF-16LE","WCHAR_T");
    if (iconvdesc == (iconv_t) -1) {
        perror("error: conversion from wchar_t to UTF-16LE is not available");
        exit(EXIT_FAILURE);
    }
    FILE * fp = fopen(tfile,"a");
    if (!fp) {
        fprintf(stderr,"error: cannot open \"%s\" for append\n",tfile,stderr);
        perror(NULL);
        exit(EXIT_FAILURE);
    }

    // Do the conversion.
    conversions = 
    iconv(iconvdesc, (char **)&inp, &remain_in, (char **)&outp, &remain_out);
    if (conversions == (size_t)-1) {
        perror("error: iconv() failed");
        exit(EXIT_FAILURE);
    }
    assert(remain_in == 0);
    // Write the UTF-16LE
    written = fwrite(converted,1,MAXOUT - remain_out,fp);
    assert(written == MAXOUT - remain_out);
    fclose(fp);
    iconv_close(iconvdesc);
    exit(EXIT_SUCCESS);
}

For GCC, wchar_t is 4 bytes wide, hence wide enough for any UTF-16. For Microsoft's compilers it is 2-bytes wide.

Documentation of <iconv.h> is here

Upvotes: 1

Related Questions