Linux Ask!

Linux Ask! is a Q & A web site specific for Linux related questions. Questions are collected, answered and audited by experienced Linux users.

How to remove duplicated lines in a file

Answer:

Use the following command

sort test.txt | uniq

  1. How to count the number of duplicated lines in a file
  2. Remove empty lines in a file
  3. Remove comments (#) and empty lines in a file
  4. Remove specified lines from a file using sed
  5. Join lines on a common field

One Response to “How to remove duplicated lines in a file”

  1. [...] You would need to sort the file and make each column an unique line first. Share and [...]

Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>