Our main server (hosts www.steeds.com, our email, file server, etc.) recently choked and had serious fsck issues when I rebooted. /var/log was no longer a directory... Ack! I was able to find most of the files in /vat/lost+found, but I decided it was time to upgrade to a version which supported journaling (in my case ext3 filesystems, though I imagine that rieserfs, jfs, xfs would all work as well). I also switched to more stable (but slower) hardware (Athlons have given me tons of heat problems).
Also, a central logserver is a good thing. Make it the most secure and stable host you've got, and a bit of disk space helps too (a GB or two is plenty!). You'll need to change the default syslogd command-line in your rc script to allow incoming syslog messages from certain hosts, open the appropriate port in your firewall (if running local to that host), and change the /etc/syslog.conf on the other hosts (there's even a util for Windows that will forward it's log events to a syslog server).
Journaling is your friend (so is central syslog server)
Our main server (hosts www.steeds.com, our email, file server, etc.)
recently choked and had serious fsck issues when I rebooted. /var/log
was no longer a directory... Ack! I was able to find most of the files
in /vat/lost+found, but I decided it was time to upgrade to a version
which supported journaling (in my case ext3 filesystems, though I imagine
that rieserfs, jfs, xfs would all work as well). I also switched
to more stable (but slower) hardware (Athlons have given me tons of heat
problems).
Also, a central logserver is a good thing. Make it the most secure and
stable host you've got, and a bit of disk space helps too (a GB or two
is plenty!). You'll need to change the default syslogd command-line in
your rc script to allow incoming syslog messages from certain hosts,
open the appropriate port in your firewall (if running local to that host), and change the /etc/syslog.conf on the other hosts (there's even a util for Windows that will forward it's log events to a syslog
server).
- Vicky