CODEWITHSUNDEEP
windowsutf-8batch-filecmdutf-16
Grzenio
Grzenio

Reputation: 36679

UTF-16 to UTF-8 conversion (for scripting in Windows)

what is the best way to convert a UTF-16 files to UTF-8? I need to use this in a cmd script.

Upvotes: 16

Views: 40947

Answers (6)

Ben Collins
Ben Collins

Reputation: 20686

You can do this easily with built-in PowerShell cmdlets, which you can invoke from cmd:

C:\> powershell -c "Get-Content mytext.txt | Set-Content -Encoding utf8 mytext_utf8.txt"

Edit: obviously if you're already in powershell, this would be simplified. Using aliases would also simplify things:

> gc mytext.txt | sc -Encoding utf8 mytext_utf8.txt

Upvotes: 8

Kaarel
Kaarel

Reputation: 10672

There is a GNU tool recode which you can also use on Windows. E.g.

recode utf16..utf8 text.txt

Upvotes: 25

Jon Skeet
Jon Skeet

Reputation: 1503439

An alternative to Ruby would be to write a small .NET program in C# (.NET 1.0 would be fine, although 2.0 would be simpler :) - it's a pretty trivial bit of code. Were you hoping to do it without any other applications at all? If you want a bit of code to do it, add a comment and I'll fill in the answer...

EDIT: Okay, this is without any kind of error checking, but...

using System;
using System.IO;
using System.Text;

class FileConverter
{
  static void Main(string[] args)
  {
    string inputFile = args[0];
    string outputFile = args[1];
    using (StreamReader reader = new StreamReader(inputFile, Encoding.Unicode))
    {
      using (StreamWriter writer = new StreamWriter(outputFile, false, Encoding.UTF8))
      {
        CopyContents(reader, writer);
      }
    }
  }

  static void CopyContents(TextReader input, TextWriter output)
  {
    char[] buffer = new char[8192];
    int len;
    while ((len = input.Read(buffer, 0, buffer.Length)) != 0)
    {
      output.Write(buffer, 0, len);
    }
  }
}

Upvotes: 15

PhiLho
PhiLho

Reputation: 41162

Perhaps with iconv?

Upvotes: 7

VonC
VonC

Reputation: 1329022

If you have a ruby distribution installed, you can call a ruby script taking care of the conversion:

Ruby script to convert file(s) character encoding

In the same spirit: Perl script

In the absence of script support, you would have to code it like this C++ source using a WideCharToMultiByte() call...

Upvotes: 1

Tor Haugen
Tor Haugen

Reputation: 19637

Certainly, the easiest way is to load the script into notepad, then save it again with the UTF-8 encoding. It's an option in the Save As dialog box..

Upvotes: 9

Related Questions