CODEWITHSUNDEEP
perlutf-8character-encoding
Pavel Vlasov
Pavel Vlasov

Reputation: 3465

How to use utf8 encode with open pragma

I have problem with utf8::encode when use pragma use open qw(:std :utf8);

Example

#!/usr/bin/env perl

use v5.16;
use utf8;
use open qw(:std :utf8);

use Data::Dumper;

my $word = "+банк";
say Dumper($word);
say utf8::is_utf8($word) ? 1 : 0;

utf8::encode($word);
say Dumper($word);
say utf8::is_utf8($word) ? 1 : 0;

Output

$VAR1 = "+\x{431}\x{430}\x{43d}\x{43a}";
1
$VAR1 = '+банк';
0

When I remove this pragma use open qw(:std :utf8);, everything is OK.

$VAR1 = "+\x{431}\x{430}\x{43d}\x{43a}";
1
$VAR1 = '+банк';
0

Thank you in advanced!

Upvotes: 2

Views: 320

Answers (2)

ysth
ysth

Reputation: 98398

utf8::encode is not what you want if you are going to print to a filehandle upon which perl expects to output utf8.

utf8::encode says take this string and give me a string where each character is a byte of the utf8 encoding of the input string. This would normally be only done if you are then going to use that string in some way where perl won't be automatically converting to utf8 if necessary.

If you add a say length($word); after the encode, you will see that $word is 9 characters, not the original 5.

Upvotes: 2

ikegami
ikegami

Reputation: 385764

If you're going to replace utf8::encode($word); with use open qw(:std :utf8);, you'll actually need to remove the utf8::encode($word);. In the version that doesn't work, you're encoding twice.

Upvotes: 6

Related Questions