1

best tool for searching within unstructured log files [closed]

view story
linux-howto

http://serverfault.com – i am supporting a number of bespoke applications at the minute and searching through their very non standard logs is a nightmare, so im looking for a tool which can do the following - load large text files search through multiple files at once and display all results can search with regex can be used to view and search unstructured text files There are some great looking log tools available but they all seem to be focused on structured logs, where the logs i deal with most days are just flat text files. I am currently using notepad++ but that has its own annoyances so im hoping there is (HowTos)

xskl's picture
Submitted by xskl on 10/22/2012

Stories similar to best tool for searching within unstructured log files [closed]

I've found out this is not as easy as I thought at first:on the other OS we know,all text files end with the .txt extension,which makes searching for them quite straightforward-no such thing on Linux,so how can I search for text files in a directory where they may be (and actually are) intermixed with all kinds of other files ?

I'm looking for pointers from anyone who's been using perl for a while on how to do a recursive text search through a massive directory using perl and when searching for a regex, not a simple string (since grep -lR can just do that). Ideally it should be perl, since once it's effective enough at locating the files I'm looking for, I'll add functions which make it do something to the results.

I'm trying to figure out the best and fastest way of finding files whose content is specifically "something"

Here's the reason: I have a process that I'm pretty sure stores its PID somewhere in a file, but I don't know where (and I want to find out). I have tried looking for filenames that may be used for storing said pid (/var/run/myproc.pid, and so on), but without success so far.

tmp files automatically saved 36 weeks 3 days ago

I have Fedora 17 GNOME installed on my Asus 1015PX. I have my personal files on a flashdrive. After closing a new or edited text files, tmp files are permanently saved on my flashdrive. I am manually opening up my folders to search for the tmp files and manually deleting the tmp files.

Hello Esteemed Members,

I need to write a script to search for files that have one or more than one rows similar. Please note that there is no specific pattern that I am searching for.

I have a directory with 1,242,276 plain text files on my external hard drive. They are totally unorganized with meaningless numbers as their file names. The reason for that is because they were recovered by some recovery software after accidentally emptying a partition a while ago.

I am now under that directory and want to search for a word "polyhedron" among the content of all the files.

I have a Java app running on a remote server (debian linux). The app logs runtime informations to files.

Once in a while, the support needs to analyze the logs using a client tool (not running on the server, but on the support's desktop).

Not sure if there are some people here who know about Recoll, a desktop search tool. A while ago, I ran Recoll which created something which I think is a database, so that I could quickly search among all the text information in the contents of all files (text file, pdf, djvu, html, etc) in one partition.

I've been using this to search an entire directory recursively for a specific phrase in my code (html, css, php, javascript, etc.):

Code: find dir_name -type f -exec grep -l "phrase" {} \; The problem is that it searches ALL files in the directory 'dir_name', even binary ones such as large JPEG images.