File access after fsck very slow for first-time access?

view story

http://unix.stackexchange.com – My drive started to show up errors, I unmounted it and ran fsck on it. It's a 2TB drive installed on Debian Squeeze, using ext3. The fsck too about 17 hours. I found some, luckily mostly unimportant, files in lost+found. But when I tried to access anything on that FS, first time access (reading directory listing, reading file information, reading file data) is very slow and takes up to one or two seconds. Accessing anything a second time seems to be normal. Given that I've around 1.5TB data on it, that's a lot times for "first access after mount". However, I don't see anything in the logs (HowTos)