CODEWITHSUNDEEP
vimduplicates
Brian Carper
Brian Carper

Reputation: 72946

How can I mark/highlight duplicate lines in VI editor?

How would you go about marking all of the lines in a buffer that are exact duplicates of other lines? By marking them, I mean highlighting them or adding a character or something. I want to retain the order of the lines in the buffer.

Before:

foo
bar
foo
baz

After:

foo*
bar
foo*
baz

Upvotes: 87

Views: 51283

Answers (6)

user7989979
user7989979

Reputation: 211

  1. :sort and save it in file1.
  2. :sort u and save it in file2.
  3. gvimdiff or tkdiff the two files.

Upvotes: 21

Krisztián Balla
Krisztián Balla

Reputation: 20371

None of the answers above worked for me so this is what I do:

  1. Sort the file using :sort
  2. Execute command :g/^\(.*\)$\n\1$/p

Upvotes: 87

rampion
rampion

Reputation: 89093

As an ex one-liner:

:syn clear Repeat | g/^\(.*\)\n\ze\%(.*\n\)*\1$/exe 'syn match Repeat "^' . escape(getline('.'), '".\^$*[]') . '$"' | nohlsearch

This uses the Repeat group to highlight the repeated lines.

Breaking it down:

  • syn clear Repeat :: remove any previously found repeats
  • g/^\(.*\)\n\ze\%(.*\n\)*\1$/ :: for any line that is repeated later in the file
    • the regex
      • ^\(.*\)\n :: a full line
      • \ze :: end of match - verify the rest of the pattern, but don't consume the matched text (positive lookahead)
      • \%(.*\n\)* :: any number of full lines
      • \1$ :: a full line repeat of the matched full line
    • exe 'syn match Repeat "^' . escape(getline('.'), '".\^$*[]') . '$"' :: add full lines that match this to the Repeat syntax group
      • exe :: execute the given string as an ex command
      • getline('.') :: the contents of the current line matched by g//
      • escape(..., '".\^$*[]') :: escape the given characters with backslashes to make a legit regex
      • syn match Repeat "^...$" :: add the given string to the Repeat syntax group
  • nohlsearch :: remove highlighting from the search done for g//

Justin's non-regex method is probably faster:

function! HighlightRepeats() range
  let lineCounts = {}
  let lineNum = a:firstline
  while lineNum <= a:lastline
    let lineText = getline(lineNum)
    if lineText != ""
      let lineCounts[lineText] = (has_key(lineCounts, lineText) ? lineCounts[lineText] : 0) + 1
    endif
    let lineNum = lineNum + 1
  endwhile
  exe 'syn clear Repeat'
  for lineText in keys(lineCounts)
    if lineCounts[lineText] >= 2
      exe 'syn match Repeat "^' . escape(lineText, '".\^$*[]') . '$"'
    endif
  endfor
endfunction

command! -range=% HighlightRepeats <line1>,<line2>call HighlightRepeats()

Upvotes: 120

Lonecat
Lonecat

Reputation: 2707

Why not use:

V*

in normal mode.

It simply searches all matches of current line, thus highlighting them (if the setting is enabled, which I think it's the default) Besides, you can then use

n

To navigate through the matches

Upvotes: 5

Zsolt Botykai
Zsolt Botykai

Reputation: 51643

Try:

:%s:^\(.\+\)\n\1:\1*\r\1:

Hope this works.

Update: next try.

:%s:^\(.\+\)$\(\_.\+\)^\1$:\1\r\2\r\1*:

Upvotes: 2

Justin
Justin

Reputation: 9801

Run through the list once, make a map of each string and how many times it occurs. Loop through it again, and append your * to any string that has a value of more than one in the map.

Upvotes: 2

Related Questions