How to remove duplicate ID's?

what the best way to find duplicate IP from file ( I have ksh script in this script I need to write function that check for duplicate IP )

for example if IP - 192.1926.23.52 exists twice in file - config_IP.txt then need to print "duplicate IP - xxx.xxx.xxx.xxx"

solution can be with ksh or perl one liner , sed ,awk etc

the format of the file should be as the following:

more config_IP.txt

I have a comma-separated list of domain-names in a file; unfortunately there are duplicate entries for some domain-names.

Hi,

I have a requirement where I have to identify duplicates from a file based on the first 6 chars (It is fixed width file of 12 chars length) and whenever a duplicate row is found, its original and duplicate row's last 2 chars should be updated to all 0's if they are not same.

Is there a program for linux which can show me a list of all duplicate music files in a directory? This will allow me to delete all duplicate files without searching for them manually. The more advanced the better;).

I have nginx setup, however I'm needing to fix an issue where someone types "/Home" to:

rewrite it to the lowercase "/home" (do this to all files and directories, not just home) but locate the Uppercase (if there is one) file. Or do I need to add another location that uses the case-insensitive characters "~*" ?

The actual directory and file layout does contain mixed Upper and lower case charac

Hello,

I have one requirement. We are getting fixed length file with trailer record on it. We need to cleanse,validate and process the file.

As part of validation we have to do duplicate file check based on file name and trailer record.

I want to add the file name as last column to file, so that I can use the filed for validation purpose.

Hi all,

I am looking for a Linux app that can find and remove duplicate images (with different filenames if that's at all possible).

Someone suggested fdupes but I'm interested in alternatives.

Thanks.

Updating Duplicate, that was the wrong duplicate Tip number is all.

← Older revision Revision as of 00:12, January 8, 2011

Line 1: Line 1:

{{review}}

{{review}}

-

{{Duplicate|743}}

+

{{Duplicate|744}}

{{TipImported

{{TipImported

|id=747

|id=747

I have a lot CSV files that I have combined. However, there are duplicates, but the entire line is not duplicated. I do have a column that I want to use as the criteria to search for a duplicate.