CODEWITHSUNDEEP
file-iofortran
DSexplorer
DSexplorer

Reputation: 365

Remove duplicate rows in a text file

How do I remove duplicate rows of string while reading a .txt file using Fortran? This is my code currently and I'm headed on a really wrong path. Currently, I'm trying to hold the first line constant for example and then comparing it to the lines after it. 

PROGRAM REM_DUP
IMPLICIT NONE
CHARACTER(632) :: ROW3, ROW4
INTEGER :: I

OPEN(UNIT=23, FILE="APM_FORMATTED.TXT", ACTION="READ", STATUS="OLD")
OPEN(UNIT=25, FILE="APM_DUPLICATES.TXT", ACTION="WRITE", STATUS="NEW")

DO
READ(23,'(A632)', END=199) ROW3
I=1
OPEN(UNIT=24, FILE="APM_FORMATTED1.TXT", ACTION="READWRITE", ACCESS="APPEND", STATUS="OLD")
    DO
    READ(24,'(A632)', END=299) ROW4
        IF(ROW3(33:52).EQ.ROW4(33:52)) THEN
        I=I+1
            IF (I.GE.3) THEN
            WRITE(25,'(A632)') ROW3
            ENDIF
        ELSE 
        WRITE(24, '(A632)') ROW3
        ENDIF
    ENDDO
    CLOSE(24)
ENDDO

199 CLOSE(23) 299 CLOSE(24) CLOSE(25)

END PROGRAM REM_DUP

Upvotes: 1

Views: 1079

Answers (1)

Kyle Kanos
Kyle Kanos

Reputation: 3264

The following might be horrendously slow, but it should work.

i=1
READ(23,'(A632)') row3
WRITE(24,'(A632)') row3 ! assume first read was unique (pretty good assumption)
DO
   READ(23,'(A632)',IOSTAT=ierr) row3
! a successful read returns ierr=0; end-of-file returns -1
   IF(ierr/=0) EXIT
! make sure we are reading from the top of the file
   REWIND(24)
   flag=.false.
! loop through file 24 for comparing
   DO k=1,i
      READ(24,'(A632)') row4
! if the line is repeated, write row3 to bad file (?) & set flag as true
      IF(row3(33:52)==row4(33:52)) THEN
         WRITE(25,'(A632)') row3
         flag = .true.
      ENDIF
   ENDDO
! if row3 is not repeated it, add to file 24 & increment i
   IF(.not.flag) THEN
      WRITE(24,'(A632)') row3
      i=i+1
   ENDIF
ENDDO
CLOSE(24); CLOSE(23); CLOSE(25)

Hopefully the comments are enough to understand.

Upvotes: 2

Related Questions