ITEM: K9623L

System down with a dirty superblock on /var.

Question:

ENVIRONMENT:  RISC model 550, AIX level 3.2.

DESCRIPTION:  When I arrived this morning, the system was dead and
there was a 513 LED on the front panel.  I booted up in maintenance
mode and got a message that there is a dirty superblock on /dev/hd9var
which is /var.  The instructions were to run a complete fsck to
fix the problem.  

Response:

We ran:

\# fsck -y /dev/hd9var

We got errors with the inode map in phase 5.  They were reported as
salvaged, but when we ran the command again, we get the same results.
Customer was booted in service mode, had run getrootfs hdisk0.  We
rebooted again and ran the getrootfs with the shell option:

\# getrootfs hdisk0 sh

When we got in, tried the fsck again, on /dev/hd9var.  We got the
same errors.  All other FS's are fsck'ed without any problems.  We
exited out, and all FS's except /var were mounted.  We received the
above error messages concerning a dirty superblock, so decided to
check on it.  We ran

\# /usr/sbin/hdf /dev/hd9var 1000 100

and saw that the magic number for the superblock was fine, this is not
a problem.  The corruption in the FS, although showing as salvaged
with fsck, is not getting salvaged, and the FS is not mountable.  It
seems our only option is to restore this FS (File System).

To recreate the FS and to restore /var:

  1.  All of the filesystems except /var are mounted
  2.  rmfs /var
  3.  mklv -L /var -t jfs -y hd9var -a m rootvg 54
  4.  mkfs -l /var -V jfs /dev/hd9var
  5.  mount -o log=/dev/hd8 /dev/hd9var /var

     The data is stored as backup by inode.  The /var backup
     is the second image on the tape.  The customer did not know
     what block size was used.  Since we can only change device
     once without losing the odm entry, I am using tctl as front end 
     to the restbyinode.  The tctl command is not affected so much
     by block size.

  6.  tctl rewind
  7.  tctl fsf 1
  8.  cd /var
  9.  tctl -b0 -Bnf /dev/rmt0.1 read | restbyinode -Bxvf-
       After about 40 minutes of restoring it failed with a media
       error.  (I think we got far enough)
  10.  sync;sync;sync;
  11.  cd /etc
       vi filesystems
       The line for /var is missing, we are going to add that line.

     /var:
        dev             = /dev/hd9var
        vfs             = jfs
        log             = /dev/hd8
        mount           = automatic
        check           = false
        type            = bootfs
        vol             = /var
        free            = false

  12. sync;sync;sync;
  13. turn key to normal
  14. shutdown -Fr
      
The system is booting up fine now.
  



Support Line: System down with a dirty superblock on /var. ITEM: K9623L

Dated: July 1994 Category: N/A

This HTML file was generated 99/06/24~13:30:41

Comments or suggestions?
Contact us