CODEWITHSUNDEEP
bashperlshellunix
Slavatron
Slavatron

Reputation: 2358

Unix - Compile a single column from many files into a single, tab-delimited file

I have a large number of files with the same, tab-delimited format:

Column A    Column B
Data_A1      Data_B1
Data_A2      Data_B2
Data_A3      Data_B3

These files all have the same number of lines.

I want to compile every file's Column B data into a single tab-delimited file. Right now, my best plan is to write a Perl script along these lines:

#!/usr/bin/perl

my $file = shift @ARGV;
my $ref = shift @ARGV;
open ( FILE, $file ); # FILE WITH FORMAT DESCRIBED ABOVE
while (<FILE>) {
        chomp;
        my @a = split("\t", $_);
        push(@B, $a[1]);
}
close FILE;

my $counter = 0;
open (REF, $ref); # TAB-DELIMITED COMPILATION OF EVERY FILE'S COLUMN B
while (<REF>) {
        chomp;
        print "$_\t$B[$counter]\n";
}
close REF;

Then, write a BASH script that loops through all the files and saving the output of the Perl script as its input for the next iteration of the shell loop:

#!/bin/bash

for file in *.txt 
     do 
          perl Script.pl $file Infile > Temp
          mv Temp Infile
     done

But this feels like a huge amount of work for something so simple. Is there a simple Unix command that can do the same thing?

Expected Output:

File1_Column_B    File2_Column_B    File3_Column_B    ...
Data_B1           Data_B1           Data_B1           ...
Data_B2           Data_B2           Data_B2           ...
Data_B3           Data_B3           Data_B3           ...
...

Upvotes: 1

Views: 146

Answers (3)

glenn jackman
glenn jackman

Reputation: 246744

bash:

paste -d'\t' input*.txt | 
awk -F'\t' '{for (i=2; i<=NF; i+=2) printf "%s%s", $i, FS; print ""}'

This pastes all the files together, with all columns, then use awk to extract only the even-numbered columns.

Upvotes: 4

choroba
choroba

Reputation: 241768

You can do all the work in Perl:

#!/usr/bin/perl
use warnings;
use strict;

my ($result, @input) = @ARGV;        # output input1 input2...

my @table;

for my $i (0 .. $#input) {
    my $infile = $input[$i];
    open my $IN, '<', $infile or die "$infile: $!";
    while (<$IN>) {
        $table[ $. - 1 ][$i] = (split)[1];
    }
}

open my $OUT, '>', $result or die "$result: $!";
for my $row (@table) {
    print {$OUT} join("\t", @$row), "\n";
}
close $OUT;

Upvotes: 2

Tiago Lopo
Tiago Lopo

Reputation: 7959

You can use awk to select the columns you want and paste to paste them together.

Example:

paste -d '\t' <(awk '{print $2}' file1.tsv) <(awk '{print $3}' file2.tsv)

NOTE: <(command) Allows the output of your command to be used as file.

Upvotes: 1

Related Questions