CODEWITHSUNDEEP
fortran
Pete W
Pete W

Reputation: 1827

Optimizing Fortran ascii file IO

I'm working on a project where I need to write some existing data to disk as ascii. I have something that works, but the IO itself is quite expensive and I'd like to optimise it further.

The data is basically an array of reals, however some of the columns store encoded strings which need to be recast as character strings (don't ask!). The input and output of this problem are beyond my control, I am receiving this real array and need to write it out as ascii.

I know that writing the array in one go as an unformatted write is faster, but this doesn't deal with the string columns correctly. Any ideas?

Here is some example code:

program test
implicit none

integer(kind=4), parameter      :: nrows = 5000
integer(kind=4), parameter      :: ncols =  400
integer, parameter              :: real_kind = 8

integer(kind=4)                 :: i,j, handle
character(len=256)              :: value_str
character(len=1)                :: delimiter
real(kind=real_kind)            :: data(nrows,ncols)

delimiter = " "
data(:,:) = 999.999

! Some examples of the "string columns"
data(:,10) = transfer('    foo ',data(1,1))
data(:,20) = transfer('    bar ',data(1,1))

handle=10
open(handle,file="out.txt",status="replace", access="stream")

do i=1,nrows
    do j=1,ncols
        ! If this column contains encoded strings then recast
        if((j==10).or.(j==20))then 
                write(handle) delimiter
                value_str = transfer(data(i,j),value_str(1:real_kind))
                write(handle) trim(value_str)
        else
                write(value_str,*) data(i,j)
                write(handle) trim(value_str)
        endif
    enddo
    write(handle) new_line('x')
enddo
close(handle)

end program test

gfortran test.F90 -o test.x

time test.x

real 0m2.65s

user 0m2.24s

sys 0m0.04s

Edit: removed "if(j/=1)" from original test.F90 code sample in response to comment.

Upvotes: 1

Views: 126

Answers (1)

innoSPG
innoSPG

Reputation: 4656

Use the free formatting and have the system handle more for you. In this proposition, I handle the transfer beforehand and use a single loop to write the data to file. This is handy if you have only few columns of character data like the 2 in your example.

Your code will look like this

program test
implicit none

    integer(kind=4), parameter      :: nrows = 5000
    integer(kind=4), parameter      :: ncols =  400
    integer, parameter              :: real_kind = 8
    integer, parameter              :: pos1 = 10 ! I like named constants
    integer, parameter              :: pos2 = 20 ! I like named constants

    integer(kind=4)                 :: i,j, handle
    character(len=256)              :: value_str
    character(len=1)                :: delimiter
    real(kind=real_kind)            :: data(nrows,ncols)
    character(real_kind), dimension(nrows,2) :: cdata ! two columns array for 

    delimiter = " "
    data(:,:) = 999.999

    ! Some examples of the "string columns"
    data(:,pos1) = transfer('    foo ',data(1,1))
    data(:,pos2) = transfer('    bar ',data(1,1))

    handle=10
    open(handle,file="out.txt",status="replace", form="formatted")

    ! Transfer beforehand
    cdata(:,1) = transfer( data(:,pos1), cdata(1,1) )
    cdata(:,2) = transfer( data(:,pos2), cdata(1,1) )
    do i=1,nrows
        write(handle,*) data(i,1:pos1-1), cdata(i,1)&
            , data(i,pos1+1:pos2-1), cdata(i,2)&
            , data(i,pos2+1:)
    enddo
    close(handle)
end program test

and give this timing

time ./test.x

real 0m1.696s

user 0m1.661s

sys 0m0.029s

instead of

time ./test.x

real 0m2.654s

user 0m2.616s

sys 0m0.032s

On my computer

Upvotes: 2

Related Questions