Monday, 2 November 2009

removing duplicates words from a line

I have a file with each line having repetitive words. I want to remove duplicate words out of each line and print all the lines in the same order.

Input file:

bash$ cat line.txt

i am fine sriram
krishna how r u , how r u sriram , How r u jadu
thank you , Thankyou sir, thank you

Required output:

i am fine sriram
krishna how r u , sriram How jadu
thank you, Thankyou sir,

Here is the awk one-liner :


awk '{ while(++i<=NF) printf (!a[$i]++) ? $i FS : ""; i=split("",a); print "" }' line.txt


1 comment:

Tweets by @sriramperumalla