Thursday 27 September 2012

How to Repair Linux File System?

My ESX VM Linux Server haulted at single user mode ( init 1 ) , after doing a restart as a result of  file system check up failure ( fsck process).

So, how to trouble shoot it?

You will see a black screen with the following information as given below:

"This is the key: the filesystem is corrupted, probably due to an unclean shutdown. It needs to be repaired. Try entering the root password, which should

give you a '#' prompt.
Type: fsck -A -y "

My Approach to it to trouble shoot  File System corruption issue:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

1.Before executing fsck, unmount all filesystems to make sure  that  no disk i/o happens.Then do fsck, then remount or just say "init 6".
2.Tips on mounting:
a.mount -o rw /  to freshly mount a disk onto / with read and write permissions to  root filesystem.
b.mount -o remount,rw  /  to  unmount,remount the disk onto / with read adn write permissions to root user or system admin.

RHEL6.2X64_Shalni: Disk name :  /dev/sda1:

So, user command :  e2fsck -f /dev/sda1  is  ok

Is shalini's machine having /dev/sda3. let me check:

e2fsck -f /dev/sda3


Reference :

So how the hell you are gonna Surviving a Filesystem Failures? Most of time fsck (front end to ext2/ext3 utility) can fix the problem, first simply run

e2fsck - to check a Linux ext2/ext3 file system (assuming /home [/dev/sda3 partition] filesystem for demo purpose), first unmount /dev/sda3 then type

following command :
# e2fsck -f /dev/sda3
Where,

    -f : Force checking even if the file system seems clean.

Please note that If the superblock is not found, e2fsck will terminate with a fatal error. However Linux maintains multiple redundant copies of the

superblock in every file system, so you can use -b {alternative-superblock} option to get rid of this problem. The location of the backup superblock is

dependent on the filesystem's blocksize:

    For filesystems with 1k blocksizes, a backup superblock can be found at block 8193
    For filesystems with 2k blocksizes, at block 16384
    For 4k blocksizes, at block 32768.

Tip you can also try any one of the following command(s) to determine alternative-superblock locations:
# mke2fs -n /dev/sda3
OR
# dumpe2fs /dev/sda3|grep -i superblock
To repair file system by alternative-superblock use command as follows:
# e2fsck -f -b 8193 /dev/sda3

However it is highly recommended that you make backup before you run fsck command on system, use dd command to create a backup (provided that you have spare

space under /disk2)
# dd if=/dev/sda2 of=/disk2/backup-sda2.img

Links:
http://www.cyberciti.biz/tips/surviving-a-linux-filesystem-failures.html
http://www.cyberciti.biz/tips/understanding-unixlinux-filesystem-superblock.html


First of all, list all the super blocks of /dev/sda1:

Following command displays primary and backup superblock location on /dev/sda3:
# dumpe2fs /dev/sda1 | grep -i superblock

Learned that, backup superblock is at 8193,24577,40961,57345,73729,204801,221185,401489:

So, let me try with alternative superblock to recognize the filesystem which ext3 written on /dev/sda1:

Command to load alternative superblock : e2fsck -f -b 8193 /dev/sda1 -  Linux

File system is fixed now on inodes,userdata,file metadata etc..


Big problem that confused me:
Now, On VMWare Linux :  how to repair /dev/mapper/VolGroup-lv_root :
check for all mount points using cat /proc/mounts


Steps I followed from Murthy Sir:

1.In single user mode # vi /etc/fstab
2.Renamed 1 2 flags  to 0 0  for logical volume which mounted on /.This tells init to ignore "FileSystem Check and FileSystem Backup".
3.use # sync  command  to flush the buffers to hard disk.
4.Then reboot using, init 6  command.
5.If it ignores Filesystem checks and comes up, it is ok to work.
6.After Linux comes up, Again check whether all filesystems are ok leaving /.
7.How to check?
8.Again vi /etc/fstab,change  0  0  to  1  2 for  /dev/mapper/VolGroup-lv_root.
9.do  sync
10.init 6

Now, your linux ios ready for working in runlevel 5.
++++++++++++++++++++++++++++++++++++++++++++++++++++

While checking hostname on linux:
1.Make sure that /etc/hosts will have last  entry of IP as the latest entry.
<IP> <FQDN> <HostName>.

2.vi /etc/sysconfig/network.
Makure Sure,
NETWORK=yes
HOSTNAME=RHEL62X64

3."Sync"
4.init 6 -> reboots to runlevel 5.

++++++++++++++++++++++++++++++++++++++++++++++++++++


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

No comments:

Post a Comment

Tweets by @sriramperumalla