python - How can I find any repeated duplication in my file -

this question has answer here:

• how catch duplicates entries in text file in linux 3 answers

how can find if file has repeated duplication. ?

many of vi files have large number of molecular co-ordinates, , sometimes, software use duplicates molecular co-ordinates on top of first one, goes unnoticed , when start using molecule in simulations, know file had repeated co-rodinates.

using general grep, need test every line , , see if pattern found.

instead, there better approach ?

ex:

c          8.72073       15.19207       10.44503  c          9.57223       14.02835       10.59743  c         10.54225       13.88199        9.86998 

repeats in file

use sort , uniq plus sed clean output:

example:

echo -e 'a\nb\nc\na\nb' b c b  echo -e 'a\nb\nc\na\nb' | sort | uniq -c       2       2 b       1 c  echo -e 'a\nb\nc\na\nb' | sort | uniq -c | sed -re '/^\s+1\s+/d; s/^\s+[0-9]+\s+//g' b