CODEWITHSUNDEEP
perlsortingcjkkanji
user813036
user813036

Reputation: 11

Sort Japanese vocabulary field by custom kanji index

I have been using a PERL program someone created 10+ years ago that inputs Japanese text (a vocabulary list) and a custom kanji index (such as RTK or KKLC or 2k1KO or Frequency), and outputs the Japanese text based on the largest kanji index appearing in that text. The idea is put words using kanji further down the index list further down the vocabulary list. In recent months with an update to Strawberry, this program stopped working and outputs the following errors:

Use of uninitialized value $mask in vec at C:/Strawberry/perl/lib/warnings.pm line 377. Unknown PerlIO layer 'encoding' at kanji-sort-1.5.pl line 8 Unknown PerlIO layer "encoding" at C:/Strawberry/perl/lib/open.pm line 120. Unknown PerlIO layer "encoding" at C:/Strawberry/perl/lib/open.pm line 128. Unknown PerlIO layer "encoding" at C:/Strawberry/perl/lib/open.pm line 129. Use of uninitialized value $mask in vec at C:/Strawberry/perl/lib/warnings.pm line 412. Use of uninitialized value $mask in bitwise and (&) at C:/Strawberry/perl/lib/warnings.pm line 424. Name "Getopt::Long::CallBack::OVERLOAD" used only once: possible typo at C:/Strawberry/perl/lib/overload.pm line 11. Unknown PerlIO layer "encoding" at kanji-sort-1.5.pl line 21. readline() on closed filehandle KANJI at kanji-sort-1.5.pl line 22.

Is there anything I can do to make this program functional again?

Here's the program. It seems simple enough with breaking vocabulary into characters and given them an value based on kanji in it with highest index score. :

#!/usr/bin/perl -w

# $ kanji-sort --kanji kanjiorder.txt --sentence-field 2 < mydeck-exported.txt > mydeck-toimport.txt
# $Revision: 1.5 $ $Date: 2010/01/08 08:22:33 $
# http://ichi2.net/anki/wiki/ContribFugounashi

use open qw( :std :encoding(UTF-8) );
use strict;
use Getopt::Long;
use utf8;

my $kanjifile;
my $sentence_field;
GetOptions(
    'sentence-field=i'=> \$sentence_field,
    'kanji=s' => \$kanjifile
);

my %kanji;
open KANJI, "<$kanjifile";
while(<KANJI>){
    chomp;
    $_=(split /\t/)[0];
    if(exists $kanji{$_}){
    print STDERR "$0: warning: ignoring duplicate kanji: $_: $kanjifile: $.\n";
    }else{
    $kanji{$_}=$.;
    }
}

my @max;
my @lines;
while(<>){
    chomp;
    my $i=$. - 1;
    $lines[$i]=$_;
    my $sentence=(split '\t', $_)[$sentence_field];
    my @chars = split //, $sentence;
    $max[$i]=0;
    foreach my $char (@chars){
    if(($kanji{$char}) && ($kanji{$char} > $max[$i])){
        $max[$i]=$kanji{$char};
    }
    }
}

my @index = 0 .. (@max - 1);
my @sorted = sort {$max[$a] <=> $max[$b]} @index;

my $last=0;
foreach my $i (@sorted){
    my $step=$max[$i] - $last;
    $last=$max[$i];
    my $sentence=(split '\t', $lines[$i])[$sentence_field];
    my @chars = split //, $sentence;
    print "$lines[$i]\t$max[$i]\t$step\n";
}

Upvotes: 1

Views: 129

Answers (1)

k-mx
k-mx

Reputation: 687

No syntax errors in your code. Checked in Strawberry Perl (Windows) and on Linux. Fix your perl installation.

Here how you can check syntax without running script:

perl -c test.pl

Upvotes: 0

Related Questions