Linux, How to sort the lines of a file

I have an 11gb wordlist file which is already sorted as each word is on its own line.

I need to remove duplicates and lines starting from 077.

I guess I need to run sed and sort -u together but I also want a live output (display whats happening in terminal) and if possible display the time left.

Quote:

Hi Am using unix ksh..

I have a file called FILE

Code: cat FILE 11/11/2012 11/11/2012 12/11/2012 15/11/2012 need to remove the duplicates dates ( ie 11/11/2012 is present two times i need remove one duplicates date )

Code: Need outputs like this 11/11/2012 12/11/2012 15/11/2012 I have tried using awk command and uniq command i

I have file which contain many hangul characters. for example:

저 가 항 공 사

인 도 의 항 공 사

I want to use linux sort for sorting lines in file but sort doesn't work. It give me:

저 가 항 공 사 인 도 의 항 공 사

So, sort only sorts spaces. How to sort this?

A huge (up to 2 GiB) text file of mine contains about 100 exact duplicates of every line in it (useless in my case, as the file is a CSV-like data table).

What I need is to remove all the repetitions while (preferably, but this can be sacrificed for a significant performance boost) maintaining the original sequence order. In the result each line is to be unique.

Introduction When you are running out of disk space, you need to concentrate on the biggest files and folders on your disk, so you can get space quickly. The best way, is to list the first 10 folders, then go inside some of them, and find files you may or can delete, and get new free space. Commands needed There is not a single command in Linux to help us with this task, but we will use du, sort

I have 2 files, one having 2 columns, another having 1 column. The second file is sorted using sort -u. Now the task is I need to join this column with the first column of the first file, which is not sorted, so what will be the syntax? will this work? join -j 1 file2.txt sort -s -n -k 1 file1.txt?

I've the following file:

1 2 3 1 4 5 1 6 7 2 3 5 5 2 1

and I want that the file be sorted for the second column but from the largest number (in this case 6) to the smallest. I've tried with

sort +1 -2 file.dat

but it gives me the inversed order.

The results should be:

1 6 7 1 4 5 2 3 5 5 2 1 1 2 3

I have tried this shell script on a SUSE 10 server, kernel 2.6.16.60, ext3 filesystem.

The script has a line like this:

cat file | awk '{print $1" "$2" "$3}' | sort -n > result

The file's size is about 3.2G, and I get the following error message: File size limit exceeded.

In this shell, ulimit -f is unlimited.

After I change script into this:

cat file | awk '{print $1" "$2" "$3}' >tmp

I'm trying to sort the following file:

a 2 b 1 a 10

I need to get:

a 2 a 10 b 1

I know about the -kPOS[opts] option, and try to use it:

sort -k1 -k2n file

but this command gives me only:

a 10 a 2 b 1

So it sorts by the first column, but no by the second.