Categories

Friday 5 April 2013

How to rescue openvz from File System corruptions

Hello,

 Some times File System in linux server may got corrupted . Suppose that /vz partition got corrupted in OpenVZ Hardware node. Then Please follow the below the steps to fix it.

You will see the file system errors in /var/log/messages and on dmesg.
Also on issuing the command df -h on the server, it will show incorrect Disk statistics if the File system is corrupted. Some times partition will show as 100% even though there is enough free space.

If /vz partition is corrupted..

Then at first stop vz service which is openvz service by command /etc/init.d/vz stop
After that unmount /vz partition using the command
umount /vz

Some times /vz partition won't un mount properly if there is some qoata issues are there on any of the Virtual Machines.

Don't use lazy unmount for unmounting it (umount -l /vz). Even if it shows as unmounted , fsck fail to run showing error as still it is mounted.

If such issue persists, then either reboot the server and enter into single user mode and run fsck on the server if you have KVM details from the DC.

If you don't have KVM details, then open /etc/fstab using the command vi /etc/fstab
Then comment out the entry of the partition /vz on it and then reboot the server

Wait for few minutes for the server to be online. As the /vz partition is commented out , it is automatically disabled when the server boots up.

ensure that /vz is unmounted using either the command mount or df -h

After that Run the fsck command as below.

fsck -fyC /vz

Always run the command on screen (screen -S fsck)


It will take some time for the task to complete depending upon size of the partition. Once it completes check the server using commands df -h. If it shows the correct disk statistics, then the issue has been fixed.

Regards


3 comments:

  1. Is there any option to boot server on rescue mode in kvm or IPMI ?

    ReplyDelete
    Replies
    1. Please check the link http://www.centos.org/docs/4/html/rhel-sag-en-4/s1-rescuemode-boot.html to boot the the machine into rescue mode.

      Delete
  2. Please check the link http://www.centos.org/docs/4/html/rhel-sag-en-4/s1-rescuemode-boot.html to boot the the machine into rescue mode.

    ReplyDelete

Ad