5

Remove Doubles Without Sort?

view full story
linux-howto

http://www.unix.com – Hi! I have concatenated two files which are wordlists, i.e., one word per line. The new file contains some doubles, but I cannot use sort and uniq as I need to keep the sort order that it is already in, which is not alphabetical, and uniq only compares adjacent lines, and the doubles are not on adjacent lines. Is there another simple way to remove doubles without altering the sort order? Unfortunately, there is no common pattern I can use to pick them out. (HowTos)