Reading a multi-dimensional array of unknown shape

Question

I want to fetch data from a file, which can have variable size in its data content. However, the structure is quite simple. 3 columns and undefined number of rows. I figured that using the allocatable multi-dimensional array and an explicit DO loop get me solve my problem. Here is my code so far

program arraycall
    implicit none

    integer, dimension(:,:), allocatable :: array
    integer :: max_rows, max_cols, row, col

    allocate(array(row,col))

    open(10, file='boundary.txt', access='sequential', status='old', FORM='FORMATTED')

     DO row = 1, max_rows
       DO col = 1, max_cols
            READ (10,*) array (row, col)
       END DO
     END DO

     print *, array (row,col)

     deallocate(array)

 end program arraycall 

Now the problem I am facing is that I don't know how should I define these max_rows and max_cols which resonates with the fact that it's of unknown size.

Sample file would may look like

11 12 13

21 22 23

31 32 33

41 42 43

---

So I figured the way to estimate record length of the file on the fly (dynamically). Updating for the future reference to others

!---------------------------------------------------------------------
! Estimate the number of records in the inputfile
!---------------------------------------------------------------------
    open(lin,file=inputfile,status='old',action='read',position='rewind')

    loop1: do
      read(lin,*,iostat=eastat) inputline
      if (eastat < 0) then
        write(*,*) trim(inputfile),": number of records = ", numvalues
        exit loop1
      else if (eastat > 0 ) then
        stop "IO-Error!"
      end if

      numvalues=numvalues+1
    end do loop1
!-----------------------------------------------------------------------
! Read the records from the inputfile
!-----------------------------------------------------------------------
    rewind(lin)
    allocate (lon(numvalues),lat(numvalues),value(numvalues))

    do i=1,numvalues
      read(lin,*) lon(i),lat(i),value(i)
    end do

    close(lin)

Answer 1

I think you have 3 options, two of which have already been described:

1. Read the file twice. First read the number of rows, then allocate and read the values. As you said, inefficient if the I/O time is relevant.

2. As @AlexanderVogt suggested, estimate the maxmimum number of rows. You don't need to carry around this large matrix during all your code. You can define a second array and do something like (based on @AlexanderVogt code):

   allocate(array2(3,tot_rows))
   array2 = array(:, :tot_rows)
   deallocate(array)
   

   Unfortunately, I'm afraid you need 2 different arrays, as you cannot resize in-place. This also means that, for a short while, you'll be using a lot of memory if array and arrray2 are large.

3. Use linked-lists. That is the most elegant solution and allows you to read the file just once without the need to pre-allocate an array. But it's the hardest to code. Here is a simple example that works for one of the arrays. You need three linked lists or a single linked list with:

   integer, dimension(3) :: data
   

   if you want it to work with 3 columns.

Linked List code:

program LinkedList
implicit none
  integer :: i, eastat, value, numvalues
  type node
      integer :: data
      type( node ), pointer :: next
  end type node
  integer, dimension(:), allocatable :: lon
  type( node ), pointer :: head, current, previous


  nullify( head )   ! Initialize list to point to no target.

  open(10,file='data.dat',status='old',action='read', position='rewind')
  numvalues = 0 
  do 
      read(10,*,iostat=eastat) value
      if (eastat < 0) then
          write(*,*) "number of records = ", numvalues
          exit
      else if (eastat > 0 ) then
          stop "IO-Error!"
      end if
      allocate( current )
      current%data = value
      current%next => head
      head => current
      numvalues=numvalues+1
  end do 
  close(10)
! The list is read. You can now convert it into an array, if needed for
! numerical efficiency

  allocate(lon(numvalues))  
  current => head
  ! You could transverse the list this way if you hadn't kept numvalues
  !do  while ( associated( current ) )
  do i= numvalues, 1, -1
      lon(i) = current%data
      previous => current
      current => current%next
!       head => current
      deallocate(previous)
   end do


! Output the list, deallocating them after use.
print *,"lon = ", lon

end program LinkedList

Answer 2

You could define a maximally allowed number of rows and make use of iostat to check for the end of the file (or an error):

program arraycall 

  implicit none
  integer, dimension(:,:), allocatable :: array
  integer           :: row
  integer           :: stat ! Check return values
  ! Define max. values
  integer,parameter :: max_rows=1000
  integer,parameter :: max_cols=3    ! As stated in the question
  ! Total number of rows in the file
  integer           :: tot_rows

  allocate( array(max_cols,max_rows), stat=stat)
  ! Always a good idea to check the return value
  if ( stat /= 0 ) stop 'Cannot allocate memory!'

  open(10, file='boundary.txt', access='sequential', &
       status='old', FORM='FORMATTED')

  DO row = 1, max_rows
    ! You can directly read in arrays! 
    READ (10,*,iostat=stat) array(:,row)
    if ( stat > 0 ) then
      stop 'An error occured while reading the file'
    elseif ( stat < 0 ) then
      tot_rows = row-1
      print *, 'EOF reached. Found a total of ', tot_rows, 'rows.'
      exit
    endif
  END DO

  close(10)

  ! Do stuff, e.g. re-allocate the array
  print *,array(:,:tot_rows)

  deallocate(array)
end program arraycall 

iostat > 0 is an error, iostat < 0 is end-of-file (or end-of-record for some compilers).