CODEWITHSUNDEEP
matlabfile-iotextscan
HCAI
HCAI

Reputation: 2263

Importing data without headers and rearranging column data

I have a 100 .dat files like the one in the left hand pane, which I need to import without headers and then sort rows.

workspace

ANSWER for doing it manually, file by file: 

data=sortrows(data,2); #sort all columns of data via the 2nd col


fid=fopen('pole_2_TurbI_chamber_05_RSM.xy');
[x ~] = textscan (fid, '%f %f', 'HeaderLines', 4);  # reads file correctly
fclose(fid);

v(:,1)=cell2mat(x(:,1)); # convert from cell to array
v(:,2)=cell2mat(x(:,2));
v=sortrows(v,2);         # sort rows according to column 2

% fig
plot(v(:,1),-v(:,2),'ro');

How can I extend this to all the files in my directory? Perhaps giving each imported variable the file name... if possible. Regards,

Upvotes: 1

Views: 1939

Answers (1)

Barton Chittenden
Barton Chittenden

Reputation: 4416

On a posix system, an individual file can be sorted using

sort -k 2 /tmp/sortme.txt

The output will be written to stdout.

If you want to sort a group of files, you would wrap everything in a for loop:

for i in *.dat
do
    sort -k 2 $i > $i.tmpsort -k 2 
    mv $i.tmp > $i
done

(in this example, make sure that you don't have any pairs of original input files that are named x.dat and x.dat.tmp, or you'll clobber x.dat.tmp).

Here's a version written in Perl, which should be portable your system (whatever you're running...). The script strips all lines which do not start with the digits 0-9.

#! /usr/bin/perl
use strict;

sub getcol2 {
     $_[0] =~ /\d+\.?\d*\s+(-?\d+\.\d+)/;
     print "$1\n";
     return( $1 ); 
}

for my $file ( @ARGV ) {
    my $INPUT;
    my @data;
    open($INPUT, "<", $file) or die "Cannot open '$file' for input: $!";


    while( <$INPUT> ) {
         #print "$_";
         push @data, $_ if(/^\d/);
    }
    close $INPUT;
    @data = sort { getcol2( $b ) <=> getcol2( $a ) } @data;    

    my $OUTPUT;
    open( $OUTPUT, ">", $file ); 

    for my $line ( @data ) {
        print $OUTPUT $line;
    }
    close( $OUTPUT ); 
}

I've called it 'sortdata.pl', it can be called as

perl sortdata.pl *.dat

It will overwrite data files; make sure that you back up the originals.

Upvotes: 1

Related Questions